Developer docs / AI Fabric tutorials

LLM coding agent with sandboxes

  • Sandbox
  • GPU
  • Web endpoint

A coding agent is a loop: the model writes code, something runs it, and the model reads the result and tries again until it works. The hard part isn't the loop; it's the something. Run model-written code on the host and one bad attempt can take you down with it.

In this tutorial you'll wire that loop together behind a web endpoint. An LLM on a GPU generates Python, a throwaway gVisor sandbox executes it, and if it fails the error feeds back into the next attempt, up to three tries. The model and the untrusted code live in two separate containers, so a broken (or hostile) snippet never reaches the agent process.

Before you start

Make sure the AI Fabric CLI is installed and authenticated (blade setup). The agent runs the model on a single RTX A4500, so this app needs a GPU. There's nothing to build by hand: the LLM uses fabric/torch (CUDA + PyTorch + transformers) and the sandbox uses a small debian_slim image.

Define two images

Create coding_agent.py. You need two distinct environments: a heavy one for the LLM on the GPU, and a light one for the code the LLM writes. Keeping them separate is what makes it safe: the sandbox image carries only what the generated code should be allowed to touch:

coding_agent.py
import bladekit

app = bladekit.app("coding-agent")

llm_image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install("torch", "transformers", "accelerate")
)

sandbox_image = (
    bladekit.Image.debian_slim(python_version="3.12")
    .pip_install("numpy", "scipy", "sympy", "matplotlib")
)

Expose the agent as an endpoint

Stack the decorators: @app.web_endpoint makes it an HTTP route and @app.function puts it on a GPU. The model is an instruct model, so we load it through the transformers pipeline. We also add a small helper that pulls the code out of a fenced Python code block, falling back to the raw text:

Python
@app.web_endpoint(methods=["POST"], min_replicas=0, max_replicas=2)
@app.function(gpu="RTX-A4500", image=llm_image, timeout=300)
def solve(item: dict) -> dict:
    """Agent loop: LLM generates code → sandbox runs it → LLM fixes if broken."""
    import re

    from transformers import pipeline

    task = item.get("task", "print the first 10 Fibonacci numbers")
    llm = pipeline("text-generation", model="Qwen/Qwen2.5-1.5B-Instruct", device=0)

    def extract_code(text: str) -> str:
        # Prefer a fenced ```python ... ``` block; fall back to the raw text.
        m = re.search(r"```(?:python)?\s*\n(.*?)```", text, re.DOTALL)
        return (m.group(1) if m else text).strip()

Generate code each attempt

The loop runs at most three times. On the first pass it asks for code; on later passes it pastes the previous error back into the prompt and asks the model to fix it. Because Qwen is an instruct model, you feed it a chat message list, not a raw string:

Python
    history = []
    for attempt in range(3):
        instruction = f"Write Python code to: {task}\n"
        if history:
            instruction += f"Previous attempt failed with:\n{history[-1]}\nFix it.\n"
        instruction += "Output ONLY the Python code in a ```python code block."

        # Qwen is an instruct model; use the chat template for usable output.
        messages = [{"role": "user", "content": instruction}]
        raw = llm(messages, max_new_tokens=256, do_sample=False)[0]["generated_text"]
        # With chat input the pipeline returns the message list; take the reply.
        reply = raw[-1]["content"] if isinstance(raw, list) else str(raw)
        code = extract_code(reply)

Run it in a sandbox and check the result

Now execute the generated code in a fresh gVisor sandbox. If it exits cleanly, return the output and stop early. If it fails, stash the truncated stderr in history so the next attempt can learn from it:

Python
        # Run the generated code in an isolated gVisor sandbox.
        with bladekit.Sandbox.create(image=sandbox_image, timeout=30) as sb:
            sb.write_file("/work/solution.py", code.encode())
            proc = sb.exec("python", "/work/solution.py")
            proc.wait()

            if proc.returncode == 0:
                return {
                    "result": "ok",
                    "attempts": attempt + 1,
                    "output": proc.stdout.read().decode(),
                    "code": code,
                }
            history.append(proc.stderr.read().decode()[:500])

    return {"result": "failed", "attempts": 3, "last_error": history[-1] if history else ""}

A new sandbox is created on every attempt, so a failed try leaves no residue for the next one. Each run starts from a clean slate.

Deploy it

Deploy the app to get a stable HTTPS URL:

Terminal
blade deploy coding_agent.py

Give it a task

POST a task to the endpoint. Endpoints are private by default, so pass your AI Fabric token as a bearer credential (or deploy with public=True for an unauthenticated endpoint):

Terminal
curl -X POST <URL> -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"task": "compute the first 20 prime numbers"}'

The model writes code, the sandbox runs it, and you get back the output along with how many attempts it took. If the model gets it wrong, it reads its own error and retries up to three times, and even if it generates something malicious, gVisor keeps it boxed in away from the host.

Take it for a spin

Everything above in one file, ready to run.

coding_agent.py
import bladekit

app = bladekit.app("coding-agent")

llm_image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install("torch", "transformers", "accelerate")
)

sandbox_image = (
    bladekit.Image.debian_slim(python_version="3.12")
    .pip_install("numpy", "scipy", "sympy", "matplotlib")
)

@app.web_endpoint(methods=["POST"], min_replicas=0, max_replicas=2)
@app.function(gpu="RTX-A4500", image=llm_image, timeout=300)
def solve(item: dict) -> dict:
    """Agent loop: LLM generates code → sandbox runs it → LLM fixes if broken."""
    import re

    from transformers import pipeline

    task = item.get("task", "print the first 10 Fibonacci numbers")
    llm = pipeline("text-generation", model="Qwen/Qwen2.5-1.5B-Instruct", device=0)

    def extract_code(text: str) -> str:
        # Prefer a fenced ```python ... ``` block; fall back to the raw text.
        m = re.search(r"```(?:python)?\s*\n(.*?)```", text, re.DOTALL)
        return (m.group(1) if m else text).strip()

    history = []
    for attempt in range(3):
        instruction = f"Write Python code to: {task}\n"
        if history:
            instruction += f"Previous attempt failed with:\n{history[-1]}\nFix it.\n"
        instruction += "Output ONLY the Python code in a ```python code block."

        # Qwen is an instruct model, so use the chat template for usable output.
        messages = [{"role": "user", "content": instruction}]
        raw = llm(messages, max_new_tokens=256, do_sample=False)[0]["generated_text"]
        # With chat input the pipeline returns the message list; take the reply.
        reply = raw[-1]["content"] if isinstance(raw, list) else str(raw)
        code = extract_code(reply)

        # Run the generated code in an isolated gVisor sandbox.
        with bladekit.Sandbox.create(image=sandbox_image, timeout=30) as sb:
            sb.write_file("/work/solution.py", code.encode())
            proc = sb.exec("python", "/work/solution.py")
            proc.wait()

            if proc.returncode == 0:
                return {
                    "result": "ok",
                    "attempts": attempt + 1,
                    "output": proc.stdout.read().decode(),
                    "code": code,
                }
            history.append(proc.stderr.read().decode()[:500])

    return {"result": "failed", "attempts": 3, "last_error": history[-1] if history else ""}

Concepts used: Endpoint, Replica, Scale-to-zero

Contact the team

Loading the form…