Developer docs / AI Fabric tutorials
LLM coding agent with sandboxes
- Sandbox
- GPU
- Web endpoint
A coding agent is a loop: the model writes code, something runs it, and the model reads the result and tries again until it works. The hard part isn't the loop; it's the something. Run model-written code on the host and one bad attempt can take you down with it.
In this tutorial you'll wire that loop together behind a web endpoint. An LLM on a GPU generates Python, a throwaway gVisor sandbox executes it, and if it fails the error feeds back into the next attempt, up to three tries. The model and the untrusted code live in two separate containers, so a broken (or hostile) snippet never reaches the agent process.
Before you start
Make sure the AI Fabric CLI is installed and authenticated (blade setup). The agent runs the model on a single RTX A4500, so this app needs a GPU. There's nothing to build by hand: the LLM uses fabric/torch (CUDA + PyTorch + transformers) and the sandbox uses a small debian_slim image.
Define two images
Create coding_agent.py. You need two distinct environments: a heavy one for the LLM on the GPU, and a light one for the code the LLM writes. Keeping them separate is what makes it safe: the sandbox image carries only what the generated code should be allowed to touch:
import bladekit
app = bladekit.app("coding-agent")
llm_image = (
bladekit.Image.from_catalog("fabric/torch")
.pip_install("torch", "transformers", "accelerate")
)
sandbox_image = (
bladekit.Image.debian_slim(python_version="3.12")
.pip_install("numpy", "scipy", "sympy", "matplotlib")
)Expose the agent as an endpoint
Stack the decorators: @app.web_endpoint makes it an HTTP route and @app.function puts it on a GPU. The model is an instruct model, so we load it through the transformers pipeline. We also add a small helper that pulls the code out of a fenced Python code block, falling back to the raw text:
@app.web_endpoint(methods=["POST"], min_replicas=0, max_replicas=2)
@app.function(gpu="RTX-A4500", image=llm_image, timeout=300)
def solve(item: dict) -> dict:
"""Agent loop: LLM generates code → sandbox runs it → LLM fixes if broken."""
import re
from transformers import pipeline
task = item.get("task", "print the first 10 Fibonacci numbers")
llm = pipeline("text-generation", model="Qwen/Qwen2.5-1.5B-Instruct", device=0)
def extract_code(text: str) -> str:
# Prefer a fenced ```python ... ``` block; fall back to the raw text.
m = re.search(r"```(?:python)?\s*\n(.*?)```", text, re.DOTALL)
return (m.group(1) if m else text).strip()Generate code each attempt
The loop runs at most three times. On the first pass it asks for code; on later passes it pastes the previous error back into the prompt and asks the model to fix it. Because Qwen is an instruct model, you feed it a chat message list, not a raw string:
history = []
for attempt in range(3):
instruction = f"Write Python code to: {task}\n"
if history:
instruction += f"Previous attempt failed with:\n{history[-1]}\nFix it.\n"
instruction += "Output ONLY the Python code in a ```python code block."
# Qwen is an instruct model; use the chat template for usable output.
messages = [{"role": "user", "content": instruction}]
raw = llm(messages, max_new_tokens=256, do_sample=False)[0]["generated_text"]
# With chat input the pipeline returns the message list; take the reply.
reply = raw[-1]["content"] if isinstance(raw, list) else str(raw)
code = extract_code(reply)Run it in a sandbox and check the result
Now execute the generated code in a fresh gVisor sandbox. If it exits cleanly, return the output and stop early. If it fails, stash the truncated stderr in history so the next attempt can learn from it:
# Run the generated code in an isolated gVisor sandbox.
with bladekit.Sandbox.create(image=sandbox_image, timeout=30) as sb:
sb.write_file("/work/solution.py", code.encode())
proc = sb.exec("python", "/work/solution.py")
proc.wait()
if proc.returncode == 0:
return {
"result": "ok",
"attempts": attempt + 1,
"output": proc.stdout.read().decode(),
"code": code,
}
history.append(proc.stderr.read().decode()[:500])
return {"result": "failed", "attempts": 3, "last_error": history[-1] if history else ""}A new sandbox is created on every attempt, so a failed try leaves no residue for the next one. Each run starts from a clean slate.
Deploy it
Deploy the app to get a stable HTTPS URL:
blade deploy coding_agent.pyGive it a task
POST a task to the endpoint. Endpoints are private by default, so pass your AI Fabric token as a bearer credential (or deploy with public=True for an unauthenticated endpoint):
curl -X POST <URL> -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"task": "compute the first 20 prime numbers"}'The model writes code, the sandbox runs it, and you get back the output along with how many attempts it took. If the model gets it wrong, it reads its own error and retries up to three times, and even if it generates something malicious, gVisor keeps it boxed in away from the host.
Take it for a spin
Everything above in one file, ready to run.
import bladekit
app = bladekit.app("coding-agent")
llm_image = (
bladekit.Image.from_catalog("fabric/torch")
.pip_install("torch", "transformers", "accelerate")
)
sandbox_image = (
bladekit.Image.debian_slim(python_version="3.12")
.pip_install("numpy", "scipy", "sympy", "matplotlib")
)
@app.web_endpoint(methods=["POST"], min_replicas=0, max_replicas=2)
@app.function(gpu="RTX-A4500", image=llm_image, timeout=300)
def solve(item: dict) -> dict:
"""Agent loop: LLM generates code → sandbox runs it → LLM fixes if broken."""
import re
from transformers import pipeline
task = item.get("task", "print the first 10 Fibonacci numbers")
llm = pipeline("text-generation", model="Qwen/Qwen2.5-1.5B-Instruct", device=0)
def extract_code(text: str) -> str:
# Prefer a fenced ```python ... ``` block; fall back to the raw text.
m = re.search(r"```(?:python)?\s*\n(.*?)```", text, re.DOTALL)
return (m.group(1) if m else text).strip()
history = []
for attempt in range(3):
instruction = f"Write Python code to: {task}\n"
if history:
instruction += f"Previous attempt failed with:\n{history[-1]}\nFix it.\n"
instruction += "Output ONLY the Python code in a ```python code block."
# Qwen is an instruct model, so use the chat template for usable output.
messages = [{"role": "user", "content": instruction}]
raw = llm(messages, max_new_tokens=256, do_sample=False)[0]["generated_text"]
# With chat input the pipeline returns the message list; take the reply.
reply = raw[-1]["content"] if isinstance(raw, list) else str(raw)
code = extract_code(reply)
# Run the generated code in an isolated gVisor sandbox.
with bladekit.Sandbox.create(image=sandbox_image, timeout=30) as sb:
sb.write_file("/work/solution.py", code.encode())
proc = sb.exec("python", "/work/solution.py")
proc.wait()
if proc.returncode == 0:
return {
"result": "ok",
"attempts": attempt + 1,
"output": proc.stdout.read().decode(),
"code": code,
}
history.append(proc.stderr.read().decode()[:500])
return {"result": "failed", "attempts": 3, "last_error": history[-1] if history else ""}Concepts used: Endpoint, Replica, Scale-to-zero