Developer docs / AI Fabric tutorials
AI Fabric tutorials.
Real-world use cases and examples you can run on AI Fabric today: LLM serving, image and audio generation, fine-tuning, batch jobs, sandboxes and more.
Before you start
Install bladekit and connect the CLI. Every tutorial then runs with blade run or deploys with blade deploy. Each one was run end to end on AI Fabric's RTX A4500 fleet. New to the SDK? See the AI Fabric SDK reference.
pip install bladekit
blade setup --api-url https://cp.run.blade.shNo tutorials match your search.
OpenAI-compatible LLM web API
Serve a drop-in OpenAI Chat Completions API from your own GPU. A custom path (/v1/chat/completions) plus the exact response schema means the official openai SDK works unchanged. Point base_url at AI Fabric and use your AI Fabric token as the api_key.
- GPU
- Web endpoint
- Concurrency
Stateful LLM service with @app.cls
Use a Python class so your model is loaded once in __init__ and reused across calls. Cleaner than a global variable, and @app.concurrent lets one GPU serve many simultaneous requests.
- GPU
- Web endpoint
- Class
- Concurrency
Container lifecycle hooks: @build, @enter, @exit
Run setup and teardown at the right moment in a container's life. @bladekit.build bakes work into the image once, @bladekit.enter loads the model once per container, and @bladekit.exit flushes and closes handles cleanly at shutdown.
- GPU
- Class
Parametrized classes: one definition, many instances
Declare typed parameters on an @app.cls with bladekit.parameter() and one class definition deploys as many independent instances: one container pool per parameter value. Serve several models from a single class, each addressed by its parameter.
- GPU
- Class
Serve vLLM with continuous batching
Run a vLLM OpenAI-compatible server inside the container with @app.web_server. AI Fabric forwards every request to the long-lived server, and @app.concurrent lets dozens of requests arrive at once so vLLM does continuous batching, with far more throughput than one-call-at-a-time handlers.
- GPU
- Web endpoint
- Concurrency
- Image DSL
Hybrid capacity: non-interruptible floor + interruptible burst
Keep a guaranteed minimum of replicas on non-interruptible capacity (always-on, never preempted) and let traffic spikes overflow onto cheaper interruptible capacity. You get a stable baseline that survives the preemption of interruptible replicas, and you only pay the premium for the floor while bursts ride the interruptible discount.
- GPU
- Web endpoint
- Concurrency
SDXL image generation
Generate images with Stable Diffusion XL. Weights are ~7 GB, so use a Volume to cache them so cold starts after the first run are fast.
- GPU
- Volume
- Image DSL
Flux image generation
Run Flux (Black Forest Labs) models for high-quality image generation, with weights cached on a Volume to skip re-downloading on every deploy.
- GPU
- Volume
- Image DSL
Transcribe audio, then digest it with an LLM
A two-step pipeline: transcribe a short audio file with Whisper, then pipe the transcript through a light LLM (Qwen 3B) for a digest. Builds on the ready fabric/torch CUDA base (torch + transformers already there), so only ffmpeg + Whisper are added. It's a quick layer, not a multi-GB torch build.
- GPU
- Image DSL
Whisper batch transcription
Transcribe hundreds of audio files in parallel using .map(). AI Fabric fans out the work across multiple GPU containers simultaneously.
- GPU
- Batch / map
- Image DSL
Text-to-speech web endpoint
Serve a TTS model as a streaming HTTP endpoint. Clients POST text, get back an audio stream.
- GPU
- Web endpoint
- Image DSL
LoRA fine-tuning
Fine-tune a model with LoRA adapters. Checkpoints are saved to a Volume so you can resume if the run is preempted.
- GPU
- Volume
- Image DSL
Fine-tuning with checkpoints on interruptible capacity
Run a LoRA fine-tune on interruptible capacity (cheap, can be preempted). A @app.on_preempt hook checkpoints the training state when the node is reclaimed with ≥5 min notice. The re-queued run resumes from the last checkpoint, minimizing wasted GPU time.
- GPU
- Volume
- Image DSL
Batch embeddings with .map()
Embed millions of documents in parallel. AI Fabric fans out chunks across multiple GPU containers and collects results in order.
- GPU
- Batch / map
- Image DSL
Dynamic batching with @bladekit.batched
Fuse many single-item calls into one GPU invocation. @bladekit.batched auto-collects concurrent calls into a batch, runs your function once over the whole batch, and fans the per-call results back. A batch fills at max_batch_size or after wait_ms, whichever comes first.
- GPU
- Batch / map
- Concurrency
- Web endpoint
Fire-and-forget fan-out with spawn_map
spawn_map is spawn + map: launch a fan-out over an iterable and get a durable handle back immediately, without blocking on results. Do other work, then collect results later, or re-attach from another process.
- Batch / map
- Async job
Parallel Parquet processing
Process a large Parquet dataset in parallel using .starmap(). Each worker handles one file shard independently.
- Batch / map
- Image DSL
RAG over PDF documents
Build a retrieval-augmented generation (RAG) pipeline: embed a PDF corpus, store the index on a Volume, then serve Q&A over HTTP.
- GPU
- Web endpoint
- Volume
- Image DSL
Protein folding with ESMFold
Predict protein 3D structure from an amino acid sequence using ESMFold. GPU inference that would take hours on a CPU runs in seconds.
- GPU
- Volume
- Image DSL
Code interpreter sandbox
Run untrusted LLM-generated Python safely in a gVisor sandbox. The sandbox is isolated at the OS kernel level, so malicious code cannot escape.
- Sandbox
- GPU
- Image DSL
LLM coding agent with sandboxes
Build an agent that writes code, runs it in a sandbox, observes the output, and iterates, all inside an isolated gVisor container.
- Sandbox
- GPU
- Web endpoint
Nightly batch job
Run a heavy data processing job every night at 3 AM UTC. AI Fabric handles the scheduling, so no cron daemon or Kubernetes CronJob needed.
- Cron
- Volume
- Image DSL
Simple web endpoint
The simplest possible AI Fabric web endpoint: a function that accepts POST requests and returns JSON. No server to manage.
- Web endpoint
Bring your own image (Docker Hub / external registry)
Two ways to choose a container image: from_catalog(...) for AI Fabric's own catalog (fabric/torch, fabric/base…), or from_registry(...) for a public image from an ALLOWLISTED registry. This deployment allows Docker Hub official images (docker.io/library/…), docker.io/vllm/vllm-openai, nvcr.io/nvidia/… (NGC) and all of ghcr.io/…; anything else is rejected with 400 IMAGE_REGISTRY_FORBIDDEN. AI Fabric installs its runner + SDK into the external image at build time. The image's Python must match your client's (cloudpickle).
- Image DSL
Build your image from a Dockerfile
Already have a Dockerfile? Point from_dockerfile(...) at it and AI Fabric builds it on its own builders: multi-stage, ARG, COPY from a build context (.dockerignore honored) all work. AI Fabric appends its runner on top, so the result runs functions like any catalog image. The final stage's Python must match your client's (cloudpickle).
- Image DSL
Resumable long training run
Run a multi-hour training job with automatic checkpointing to a Volume. If the run is preempted, it resumes from the last checkpoint.
- GPU
- Volume
- Image DSL
Document OCR web app
POST an image and get back extracted text as JSON. Uses the Tesseract OCR engine: fast, CPU-only, no GPU or heavy ML stack required.
- Web endpoint
- Image DSL
Multimodal RAG over PDF (VLM embeddings)
Index PDF pages as images using a vision-language model (ColPali), store embeddings in a Volume, and serve a Q&A endpoint.
- GPU
- Volume
- Web endpoint
- Image DSL
Embeddings web endpoint (sentence-transformers)
Serve a fast text-embedding endpoint backed by your own GPU. The BGE model loads once in a warm @app.cls replica; POST a batch of texts to /embed and get vectors back.
- GPU
- Volume
- Web endpoint
- Class
Satellite images → embedding vectors
Embed batches of satellite-style images with a pretrained ResNet encoder on the GPU, fanned out with .map(). Self-contained: the imagery is synthesised in-code, so there's nothing to download.
- GPU
- Batch / map
- Image DSL
OCR job queue (bladekit.Queue)
Use bladekit.Queue as a job queue: a producer enqueues documents, and a worker function dequeues and processes them asynchronously.
- Image DSL
Fast Flux with torch.compile
Speed up Flux image generation by compiling the UNet with torch.compile(mode="reduce-overhead"). First call pays the compile cost; all subsequent calls are faster.
- GPU
- Volume
- Class
- Image DSL
Animate an image (video generation)
Turn a still image into a short video clip using AnimateDiff. The function returns raw MP4 bytes you can stream or save locally.
- GPU
- Volume
- Image DSL
LTX-Video text-to-video
Generate high-quality video from a text prompt with LTX-Video (Lightricks). Returns an MP4 file.
- GPU
- Volume
- Image DSL
Stable Diffusion API (CLI + web)
Expose Stable Diffusion 1.5 both as a CLI tool (via local_entrypoint) and as a web endpoint. Same function, two interfaces.
- GPU
- Volume
- Web endpoint
- Image DSL
Voice chat with an LLM
A voice chat pipeline: Whisper transcribes audio, an LLM generates a reply, and TTS converts it back to speech, all served from a single web endpoint.
- GPU
- Web endpoint
- Image DSL
Full-duplex voice chat with Moshi
Deploy Kyutai's Moshi, a real-time, full-duplex speech-to-speech model that listens and talks at the same time over a single audio stream. One @app.web_server hosts Moshi's built-in browser UI and websocket, no STT -> LLM -> TTS pipeline in between.
- GPU
- Web endpoint
- Image DSL
Music generation with ACE-Step
Generate music from a text prompt using ACE-Step, a diffusion-based audio model from HuggingFace. Returns raw audio bytes.
- GPU
- Volume
- Image DSL
TTS with Chatterbox
Serve Chatterbox (resemble-ai), a voice-cloning TTS model, with a warm @app.cls replica and Volume-cached weights.
- GPU
- Volume
- Class
- Image DSL
High-throughput Whisper with batching
Transcribe audio at high throughput by combining faster-whisper batch processing with @app.concurrent so multiple files share one GPU.
- GPU
- Batch / map
- Concurrency
- Image DSL
Streaming speech-to-text with Kyutai STT
Real-time, incremental transcription with Kyutai STT (moshi delayed-streams). Unlike batch Whisper (which needs the whole clip before it says a word), this model consumes audio frame-by-frame (80 ms Mimi codec frames) and emits text as it listens, so partial transcripts grow while the audio is still playing. Builds on the fabric/torch base; only the moshi runtime is added.
- GPU
- Image DSL
DuckDB + dbt data warehouse
Run a dbt project with a DuckDB backend on AI Fabric. Data files are stored on a Volume; the full transform pipeline runs on demand.
- Volume
- Image DSL
Mount an S3 bucket as a filesystem path
Mount an external S3/GCS/R2 bucket at a path with bladekit.CloudBucketMount and read it like local files. Credentials come only from a Secret; read_only is honored.
- Secret
- Image DSL
Publish explorable data with Datasette
Deploy Datasette as a long-lived web app on AI Fabric to make your SQLite data explorable (a browsable UI plus a JSON API) behind the app gateway. The database lives on a Volume, populated once and reused across replicas.
- Volume
- Web endpoint
- Image DSL
GRPO + veRL math reasoning (RL training)
Train an LLM to solve math problems using GRPO reinforcement learning with veRL. Uses 8 RTX-A4500s and saves checkpoints to a Volume.
- GPU
- Volume
- Image DSL
GRPO + TRL code generation RL
Fine-tune an LLM for code generation using GRPO with TRL's GRPOTrainer. A reward model scores generated code quality.
- GPU
- Volume
- Image DSL
Hyperparameter grid search with early stopping
Launch N training experiments in parallel with .starmap(). Each experiment reports its validation metric so you can select the best config.
- GPU
- Batch / map
- Volume
- Image DSL
Blender render farm
Render a Blender animation in parallel by distributing frames across many containers. Each container renders one frame; .map() collects them all.
- Batch / map
- Image DSL
Live transcription over WebRTC
Stream your microphone to an AI Fabric container over WebRTC and get a live transcript back in real time. The container runs faster-whisper; the audio (UDP) flows browser-to-container directly when the network allows, with the platform's TURN relay as fallback, while the HTTP endpoint only handles signalling.
- Web endpoint
- Image DSL
Run Node.js / Ruby in a sandbox
Install Node.js and Ruby in sandbox images, then execute scripts in isolated containers. Useful for multi-language agent tooling.
- Sandbox
- Image DSL
Stateless MCP server with FastMCP
Build a stateless MCP (Model Context Protocol) server with FastMCP, where each tool invocation runs user code in a fresh sandbox.
- Sandbox
- Web endpoint
- Image DSL
Sandboxed LangGraph agent
Build a LangGraph StateGraph agent (generate → run-in-sandbox → retry) whose only tool is a throwaway gVisor sandbox. Tested end to end: it writes code, runs it safely, and returns the first 10 primes.
- Sandbox
- GPU
- LangGraph
- Web endpoint
Stateful code interpreter (REPL)
Keep a sandbox alive across multiple code executions so variables persist between turns: a stateful REPL for LLM agents.
- Sandbox
- Image DSL
Warm pools: cutting sandbox cold-start
AI Fabric has no server-side warm-pool primitive yet. Pre-create a small pool of running sandboxes client-side and hand them out instantly instead of paying create() latency on every request.
- Sandbox
- Image DSL
Vibe coding for thousands of users
Fan out one sandbox per user request using .map() and a bladekit.Queue for back-pressure. Each user gets an isolated environment.
- Sandbox
- Batch / map
- Image DSL
OpenCode agents at scale
Launch hundreds of parallel coding agents, each running in its own sandbox. Use .map() to distribute tasks across the fleet.
- Sandbox
- Batch / map
- Image DSL
Protein binder design (ESMFold2 fan-out)
Fold hundreds of candidate binder sequences in parallel using ESMFold and .map(). Each sequence gets its own GPU container.
- GPU
- Batch / map
- Volume
- Image DSL
Fine-tune an LLM with Unsloth
Fine-tune with Unsloth's fused Triton kernels: faster training and much lower VRAM via 4-bit quantization. The trained LoRA adapter is saved to a Volume.
- GPU
- Volume
- Image DSL
Flux LoRA fine-tuning on a 20 GB GPU
Fine-tune FLUX.1-dev with LoRA on a single RTX A4500. NF4-quantised transformer plus a cache-then-free pass for the text encoders keeps the peak at 10.8 GiB of 19.6. No H100 needed.
- GPU
- Volume
- Secret
- Image DSL
Fine-tune Whisper
Fine-tune OpenAI Whisper on a custom language or domain dataset using HuggingFace Seq2SeqTrainer. Checkpoints go to a Volume.
- GPU
- Volume
- Image DSL
Fine-tune + serve YOLO
Fine-tune a YOLO model on a custom dataset, save weights to a Volume, then serve it as a web endpoint for real-time object detection.
- GPU
- Volume
- Web endpoint
- Image DSL
Pet art generator (Flux + Gradio)
Fine-tune Flux LoRA on pet photos, then serve a Gradio web UI so anyone can generate artwork of their pet.
- GPU
- Volume
- Web endpoint
- Image DSL
Tensor-parallel LLM on 8× RTX-A4500
Shard a 32B LLM across 8 RTX-A4500s with vLLM tensor parallelism. One @app.web_server replica owns all 8 GPUs of the node and serves concurrent requests with continuous batching.
- GPU
- Web endpoint
- Concurrency
- Volume
Low-latency serving with SGLang
Serve an LLM with SGLang for lower latency and higher throughput. The @app.cls pattern keeps the engine warm; @app.concurrent batches requests.
- GPU
- Class
- Concurrency
- Web endpoint
Reduce cold starts with Volume cache
Download model weights into a Volume at build time so every cold start loads from fast local storage instead of re-downloading from HuggingFace.
- GPU
- Volume
- Image DSL
Embed a whole dataset on the GPU fleet
Embed a dataset in parallel with .map(): BGE on RTX-A4500, ~25x faster than CPU, billed per started minute. Measured: 400 docs, 0.30s GPU vs 7.45s CPU.
- GPU
- Batch / map
- Image DSL
Batch image upscaling (4x) on GPU
Upscale a batch of images 4x with Swin2SR on RTX-A4500, fanned out with .map(). Measured: 256->1024 in ~1.5s/image, all in parallel (tens of seconds each on CPU).
- GPU
- Batch / map
- Image DSL
- Volume
Batch background removal on GPU
Cut backgrounds from a batch of images with RMBG-1.4 on RTX-A4500 into transparent RGBA PNGs, billed per started minute. Measured: 3/3 images at ~0.1s GPU/image, all in parallel.
- GPU
- Batch / map
- Image DSL
- Volume
Batch image captioning on GPU
Caption a batch of images with BLIP-large on RTX-A4500, fanned out with .map(): one GPU container per image, all at once. Measured: 5/5 images, ~0.35s GPU/image.
- GPU
- Batch / map
- Image DSL
- Volume
Batch translation on GPU (NLLB-200)
Translate a batch of texts with Meta's NLLB-200 on RTX-A4500, fanned out with .map(). Measured: 52.4x faster than the same model on CPU.
- GPU
- Batch / map
- Image DSL
- Volume
Batch summarization on GPU (BART)
Summarize a batch of documents with BART-large-CNN on RTX-A4500, fanned out with .map(). Measured: ~93x faster than CPU per document.
- GPU
- Batch / map
- Image DSL
- Volume
Semantic near-duplicate detection
Embed a dataset with BGE-small on RTX-A4500, fan it out with .map(), then find near-duplicate pairs by cosine similarity. Both embedding and dedup run remotely. Measured: 500 sentences, 27.7x faster than CPU.
- GPU
- Batch / map
- Image DSL
- Volume
Zero-shot text classification on GPU
Classify texts into arbitrary labels with BART-large-MNLI (zero-shot, no training) on RTX-A4500, fanned out with .map(). Measured: ~104x faster than CPU.
- GPU
- Batch / map
- Image DSL
- Volume
Async jobs: submit now, get the result later
Kick off a long job on a deployed endpoint without holding the HTTP connection: spawn() returns a call_id immediately, and you poll for the result whenever you want, even from another process.
- Async job
- Web endpoint
Fan out hundreds of async jobs, gather the results
spawn() is non-blocking, so you can fire a whole batch of jobs at once and collect them by handle. The autoscaler adds replicas to chew through the queue in parallel.
- Async job
- Web endpoint
Completion webhooks: get notified when a job finishes
Instead of polling, register a webhook: AI Fabric POSTs your URL the moment the job completes, with an HMAC signature you verify. The receiver is another AI Fabric endpoint.
- Async job
- Web endpoint
- Secret