Developer docs / AI Fabric tutorials

AI Fabric tutorials.

Real-world use cases and examples you can run on AI Fabric today: LLM serving, image and audio generation, fine-tuning, batch jobs, sandboxes and more.

Before you start

Install bladekit and connect the CLI. Every tutorial then runs with blade run or deploys with blade deploy. Each one was run end to end on AI Fabric's RTX A4500 fleet. New to the SDK? See the AI Fabric SDK reference.

Install and connect
pip install bladekit
blade setup --api-url https://cp.run.blade.sh

LLM inference & serving

OpenAI-compatible LLM web API

Serve a drop-in OpenAI Chat Completions API from your own GPU. A custom path (/v1/chat/completions) plus the exact response schema means the official openai SDK works unchanged. Point base_url at AI Fabric and use your AI Fabric token as the api_key.

  • GPU
  • Web endpoint
  • Concurrency
LLM inference & serving

Stateful LLM service with @app.cls

Use a Python class so your model is loaded once in __init__ and reused across calls. Cleaner than a global variable, and @app.concurrent lets one GPU serve many simultaneous requests.

  • GPU
  • Web endpoint
  • Class
  • Concurrency
LLM inference & serving

Container lifecycle hooks: @build, @enter, @exit

Run setup and teardown at the right moment in a container's life. @bladekit.build bakes work into the image once, @bladekit.enter loads the model once per container, and @bladekit.exit flushes and closes handles cleanly at shutdown.

  • GPU
  • Class
LLM inference & serving

Parametrized classes: one definition, many instances

Declare typed parameters on an @app.cls with bladekit.parameter() and one class definition deploys as many independent instances: one container pool per parameter value. Serve several models from a single class, each addressed by its parameter.

  • GPU
  • Class
LLM inference & serving

Serve vLLM with continuous batching

Run a vLLM OpenAI-compatible server inside the container with @app.web_server. AI Fabric forwards every request to the long-lived server, and @app.concurrent lets dozens of requests arrive at once so vLLM does continuous batching, with far more throughput than one-call-at-a-time handlers.

  • GPU
  • Web endpoint
  • Concurrency
  • Image DSL
LLM inference & serving

Hybrid capacity: non-interruptible floor + interruptible burst

Keep a guaranteed minimum of replicas on non-interruptible capacity (always-on, never preempted) and let traffic spikes overflow onto cheaper interruptible capacity. You get a stable baseline that survives the preemption of interruptible replicas, and you only pay the premium for the floor while bursts ride the interruptible discount.

  • GPU
  • Web endpoint
  • Concurrency
Image & video

SDXL image generation

Generate images with Stable Diffusion XL. Weights are ~7 GB, so use a Volume to cache them so cold starts after the first run are fast.

  • GPU
  • Volume
  • Image DSL
Image & video

Flux image generation

Run Flux (Black Forest Labs) models for high-quality image generation, with weights cached on a Volume to skip re-downloading on every deploy.

  • GPU
  • Volume
  • Image DSL
Audio & speech

Transcribe audio, then digest it with an LLM

A two-step pipeline: transcribe a short audio file with Whisper, then pipe the transcript through a light LLM (Qwen 3B) for a digest. Builds on the ready fabric/torch CUDA base (torch + transformers already there), so only ffmpeg + Whisper are added. It's a quick layer, not a multi-GB torch build.

  • GPU
  • Image DSL
Audio & speech

Whisper batch transcription

Transcribe hundreds of audio files in parallel using .map(). AI Fabric fans out the work across multiple GPU containers simultaneously.

  • GPU
  • Batch / map
  • Image DSL
Audio & speech

Text-to-speech web endpoint

Serve a TTS model as a streaming HTTP endpoint. Clients POST text, get back an audio stream.

  • GPU
  • Web endpoint
  • Image DSL
Fine-tuning

LoRA fine-tuning

Fine-tune a model with LoRA adapters. Checkpoints are saved to a Volume so you can resume if the run is preempted.

  • GPU
  • Volume
  • Image DSL
Fine-tuning

Fine-tuning with checkpoints on interruptible capacity

Run a LoRA fine-tune on interruptible capacity (cheap, can be preempted). A @app.on_preempt hook checkpoints the training state when the node is reclaimed with ≥5 min notice. The re-queued run resumes from the last checkpoint, minimizing wasted GPU time.

  • GPU
  • Volume
  • Image DSL
Batch & embeddings

Batch embeddings with .map()

Embed millions of documents in parallel. AI Fabric fans out chunks across multiple GPU containers and collects results in order.

  • GPU
  • Batch / map
  • Image DSL
Batch & embeddings

Dynamic batching with @bladekit.batched

Fuse many single-item calls into one GPU invocation. @bladekit.batched auto-collects concurrent calls into a batch, runs your function once over the whole batch, and fans the per-call results back. A batch fills at max_batch_size or after wait_ms, whichever comes first.

  • GPU
  • Batch / map
  • Concurrency
  • Web endpoint
Batch & embeddings

Fire-and-forget fan-out with spawn_map

spawn_map is spawn + map: launch a fan-out over an iterable and get a durable handle back immediately, without blocking on results. Do other work, then collect results later, or re-attach from another process.

  • Batch / map
  • Async job
Batch & embeddings

Parallel Parquet processing

Process a large Parquet dataset in parallel using .starmap(). Each worker handles one file shard independently.

  • Batch / map
  • Image DSL
RAG

RAG over PDF documents

Build a retrieval-augmented generation (RAG) pipeline: embed a PDF corpus, store the index on a Volume, then serve Q&A over HTTP.

  • GPU
  • Web endpoint
  • Volume
  • Image DSL
Science & protein folding

Protein folding with ESMFold

Predict protein 3D structure from an amino acid sequence using ESMFold. GPU inference that would take hours on a CPU runs in seconds.

  • GPU
  • Volume
  • Image DSL
Sandboxes & agents

Code interpreter sandbox

Run untrusted LLM-generated Python safely in a gVisor sandbox. The sandbox is isolated at the OS kernel level, so malicious code cannot escape.

  • Sandbox
  • GPU
  • Image DSL
Sandboxes & agents

LLM coding agent with sandboxes

Build an agent that writes code, runs it in a sandbox, observes the output, and iterates, all inside an isolated gVisor container.

  • Sandbox
  • GPU
  • Web endpoint
Crons & scheduling

Nightly batch job

Run a heavy data processing job every night at 3 AM UTC. AI Fabric handles the scheduling, so no cron daemon or Kubernetes CronJob needed.

  • Cron
  • Volume
  • Image DSL
Web apps & integrations

Simple web endpoint

The simplest possible AI Fabric web endpoint: a function that accepts POST requests and returns JSON. No server to manage.

  • Web endpoint
Web apps & integrations

Bring your own image (Docker Hub / external registry)

Two ways to choose a container image: from_catalog(...) for AI Fabric's own catalog (fabric/torch, fabric/base…), or from_registry(...) for a public image from an ALLOWLISTED registry. This deployment allows Docker Hub official images (docker.io/library/…), docker.io/vllm/vllm-openai, nvcr.io/nvidia/… (NGC) and all of ghcr.io/…; anything else is rejected with 400 IMAGE_REGISTRY_FORBIDDEN. AI Fabric installs its runner + SDK into the external image at build time. The image's Python must match your client's (cloudpickle).

  • Image DSL
Web apps & integrations

Build your image from a Dockerfile

Already have a Dockerfile? Point from_dockerfile(...) at it and AI Fabric builds it on its own builders: multi-stage, ARG, COPY from a build context (.dockerignore honored) all work. AI Fabric appends its runner on top, so the result runs functions like any catalog image. The final stage's Python must match your client's (cloudpickle).

  • Image DSL
Data & training

Resumable long training run

Run a multi-hour training job with automatic checkpointing to a Volume. If the run is preempted, it resumes from the last checkpoint.

  • GPU
  • Volume
  • Image DSL
Web apps & integrations

Document OCR web app

POST an image and get back extracted text as JSON. Uses the Tesseract OCR engine: fast, CPU-only, no GPU or heavy ML stack required.

  • Web endpoint
  • Image DSL
Web apps & integrations

Multimodal RAG over PDF (VLM embeddings)

Index PDF pages as images using a vision-language model (ColPali), store embeddings in a Volume, and serve a Q&A endpoint.

  • GPU
  • Volume
  • Web endpoint
  • Image DSL
Batch & embeddings

Embeddings web endpoint (sentence-transformers)

Serve a fast text-embedding endpoint backed by your own GPU. The BGE model loads once in a warm @app.cls replica; POST a batch of texts to /embed and get vectors back.

  • GPU
  • Volume
  • Web endpoint
  • Class
Batch & embeddings

Satellite images → embedding vectors

Embed batches of satellite-style images with a pretrained ResNet encoder on the GPU, fanned out with .map(). Self-contained: the imagery is synthesised in-code, so there's nothing to download.

  • GPU
  • Batch / map
  • Image DSL
Batch & embeddings

OCR job queue (bladekit.Queue)

Use bladekit.Queue as a job queue: a producer enqueues documents, and a worker function dequeues and processes them asynchronously.

  • Image DSL
Image & video

Fast Flux with torch.compile

Speed up Flux image generation by compiling the UNet with torch.compile(mode="reduce-overhead"). First call pays the compile cost; all subsequent calls are faster.

  • GPU
  • Volume
  • Class
  • Image DSL
Image & video

Animate an image (video generation)

Turn a still image into a short video clip using AnimateDiff. The function returns raw MP4 bytes you can stream or save locally.

  • GPU
  • Volume
  • Image DSL
Image & video

LTX-Video text-to-video

Generate high-quality video from a text prompt with LTX-Video (Lightricks). Returns an MP4 file.

  • GPU
  • Volume
  • Image DSL
Image & video

Stable Diffusion API (CLI + web)

Expose Stable Diffusion 1.5 both as a CLI tool (via local_entrypoint) and as a web endpoint. Same function, two interfaces.

  • GPU
  • Volume
  • Web endpoint
  • Image DSL
Audio & speech

Voice chat with an LLM

A voice chat pipeline: Whisper transcribes audio, an LLM generates a reply, and TTS converts it back to speech, all served from a single web endpoint.

  • GPU
  • Web endpoint
  • Image DSL
Audio & speech

Full-duplex voice chat with Moshi

Deploy Kyutai's Moshi, a real-time, full-duplex speech-to-speech model that listens and talks at the same time over a single audio stream. One @app.web_server hosts Moshi's built-in browser UI and websocket, no STT -> LLM -> TTS pipeline in between.

  • GPU
  • Web endpoint
  • Image DSL
Audio & speech

Music generation with ACE-Step

Generate music from a text prompt using ACE-Step, a diffusion-based audio model from HuggingFace. Returns raw audio bytes.

  • GPU
  • Volume
  • Image DSL
Audio & speech

TTS with Chatterbox

Serve Chatterbox (resemble-ai), a voice-cloning TTS model, with a warm @app.cls replica and Volume-cached weights.

  • GPU
  • Volume
  • Class
  • Image DSL
Audio & speech

High-throughput Whisper with batching

Transcribe audio at high throughput by combining faster-whisper batch processing with @app.concurrent so multiple files share one GPU.

  • GPU
  • Batch / map
  • Concurrency
  • Image DSL
Audio & speech

Streaming speech-to-text with Kyutai STT

Real-time, incremental transcription with Kyutai STT (moshi delayed-streams). Unlike batch Whisper (which needs the whole clip before it says a word), this model consumes audio frame-by-frame (80 ms Mimi codec frames) and emits text as it listens, so partial transcripts grow while the audio is still playing. Builds on the fabric/torch base; only the moshi runtime is added.

  • GPU
  • Image DSL
Data warehouse

DuckDB + dbt data warehouse

Run a dbt project with a DuckDB backend on AI Fabric. Data files are stored on a Volume; the full transform pipeline runs on demand.

  • Volume
  • Image DSL
Data warehouse

Mount an S3 bucket as a filesystem path

Mount an external S3/GCS/R2 bucket at a path with bladekit.CloudBucketMount and read it like local files. Credentials come only from a Secret; read_only is honored.

  • Secret
  • Image DSL
Data warehouse

Publish explorable data with Datasette

Deploy Datasette as a long-lived web app on AI Fabric to make your SQLite data explorable (a browsable UI plus a JSON API) behind the app gateway. The database lives on a Volume, populated once and reused across replicas.

  • Volume
  • Web endpoint
  • Image DSL
RL & training

GRPO + veRL math reasoning (RL training)

Train an LLM to solve math problems using GRPO reinforcement learning with veRL. Uses 8 RTX-A4500s and saves checkpoints to a Volume.

  • GPU
  • Volume
  • Image DSL
RL & training

GRPO + TRL code generation RL

Fine-tune an LLM for code generation using GRPO with TRL's GRPOTrainer. A reward model scores generated code quality.

  • GPU
  • Volume
  • Image DSL
RL & training

Hyperparameter grid search with early stopping

Launch N training experiments in parallel with .starmap(). Each experiment reports its validation metric so you can select the best config.

  • GPU
  • Batch / map
  • Volume
  • Image DSL
3D rendering

Blender render farm

Render a Blender animation in parallel by distributing frames across many containers. Each container renders one frame; .map() collects them all.

  • Batch / map
  • Image DSL
Web apps & integrations

Live transcription over WebRTC

Stream your microphone to an AI Fabric container over WebRTC and get a live transcript back in real time. The container runs faster-whisper; the audio (UDP) flows browser-to-container directly when the network allows, with the platform's TURN relay as fallback, while the HTTP endpoint only handles signalling.

  • Web endpoint
  • Image DSL
Sandboxes & agents

Run Node.js / Ruby in a sandbox

Install Node.js and Ruby in sandbox images, then execute scripts in isolated containers. Useful for multi-language agent tooling.

  • Sandbox
  • Image DSL
Sandboxes & agents

Stateless MCP server with FastMCP

Build a stateless MCP (Model Context Protocol) server with FastMCP, where each tool invocation runs user code in a fresh sandbox.

  • Sandbox
  • Web endpoint
  • Image DSL
Sandboxes & agents

Sandboxed LangGraph agent

Build a LangGraph StateGraph agent (generate → run-in-sandbox → retry) whose only tool is a throwaway gVisor sandbox. Tested end to end: it writes code, runs it safely, and returns the first 10 primes.

  • Sandbox
  • GPU
  • LangGraph
  • Web endpoint
Sandboxes & agents

Stateful code interpreter (REPL)

Keep a sandbox alive across multiple code executions so variables persist between turns: a stateful REPL for LLM agents.

  • Sandbox
  • Image DSL
Sandboxes & agents

Warm pools: cutting sandbox cold-start

AI Fabric has no server-side warm-pool primitive yet. Pre-create a small pool of running sandboxes client-side and hand them out instantly instead of paying create() latency on every request.

  • Sandbox
  • Image DSL
Sandboxes & agents

Vibe coding for thousands of users

Fan out one sandbox per user request using .map() and a bladekit.Queue for back-pressure. Each user gets an isolated environment.

  • Sandbox
  • Batch / map
  • Image DSL
Sandboxes & agents

OpenCode agents at scale

Launch hundreds of parallel coding agents, each running in its own sandbox. Use .map() to distribute tasks across the fleet.

  • Sandbox
  • Batch / map
  • Image DSL
Science & protein folding

Protein binder design (ESMFold2 fan-out)

Fold hundreds of candidate binder sequences in parallel using ESMFold and .map(). Each sequence gets its own GPU container.

  • GPU
  • Batch / map
  • Volume
  • Image DSL
Fine-tuning

Fine-tune an LLM with Unsloth

Fine-tune with Unsloth's fused Triton kernels: faster training and much lower VRAM via 4-bit quantization. The trained LoRA adapter is saved to a Volume.

  • GPU
  • Volume
  • Image DSL
Fine-tuning

Flux LoRA fine-tuning on a 20 GB GPU

Fine-tune FLUX.1-dev with LoRA on a single RTX A4500. NF4-quantised transformer plus a cache-then-free pass for the text encoders keeps the peak at 10.8 GiB of 19.6. No H100 needed.

  • GPU
  • Volume
  • Secret
  • Image DSL
Fine-tuning

Fine-tune Whisper

Fine-tune OpenAI Whisper on a custom language or domain dataset using HuggingFace Seq2SeqTrainer. Checkpoints go to a Volume.

  • GPU
  • Volume
  • Image DSL
Fine-tuning

Fine-tune + serve YOLO

Fine-tune a YOLO model on a custom dataset, save weights to a Volume, then serve it as a web endpoint for real-time object detection.

  • GPU
  • Volume
  • Web endpoint
  • Image DSL
Fine-tuning

Pet art generator (Flux + Gradio)

Fine-tune Flux LoRA on pet photos, then serve a Gradio web UI so anyone can generate artwork of their pet.

  • GPU
  • Volume
  • Web endpoint
  • Image DSL
LLM inference & serving

Tensor-parallel LLM on 8× RTX-A4500

Shard a 32B LLM across 8 RTX-A4500s with vLLM tensor parallelism. One @app.web_server replica owns all 8 GPUs of the node and serves concurrent requests with continuous batching.

  • GPU
  • Web endpoint
  • Concurrency
  • Volume
LLM inference & serving

Low-latency serving with SGLang

Serve an LLM with SGLang for lower latency and higher throughput. The @app.cls pattern keeps the engine warm; @app.concurrent batches requests.

  • GPU
  • Class
  • Concurrency
  • Web endpoint
LLM inference & serving

Reduce cold starts with Volume cache

Download model weights into a Volume at build time so every cold start loads from fast local storage instead of re-downloading from HuggingFace.

  • GPU
  • Volume
  • Image DSL
Batch & embeddings

Embed a whole dataset on the GPU fleet

Embed a dataset in parallel with .map(): BGE on RTX-A4500, ~25x faster than CPU, billed per started minute. Measured: 400 docs, 0.30s GPU vs 7.45s CPU.

  • GPU
  • Batch / map
  • Image DSL
Image & video

Batch image upscaling (4x) on GPU

Upscale a batch of images 4x with Swin2SR on RTX-A4500, fanned out with .map(). Measured: 256->1024 in ~1.5s/image, all in parallel (tens of seconds each on CPU).

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Image & video

Batch background removal on GPU

Cut backgrounds from a batch of images with RMBG-1.4 on RTX-A4500 into transparent RGBA PNGs, billed per started minute. Measured: 3/3 images at ~0.1s GPU/image, all in parallel.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Image & video

Batch image captioning on GPU

Caption a batch of images with BLIP-large on RTX-A4500, fanned out with .map(): one GPU container per image, all at once. Measured: 5/5 images, ~0.35s GPU/image.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Batch & embeddings

Batch translation on GPU (NLLB-200)

Translate a batch of texts with Meta's NLLB-200 on RTX-A4500, fanned out with .map(). Measured: 52.4x faster than the same model on CPU.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Batch & embeddings

Batch summarization on GPU (BART)

Summarize a batch of documents with BART-large-CNN on RTX-A4500, fanned out with .map(). Measured: ~93x faster than CPU per document.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Batch & embeddings

Semantic near-duplicate detection

Embed a dataset with BGE-small on RTX-A4500, fan it out with .map(), then find near-duplicate pairs by cosine similarity. Both embedding and dedup run remotely. Measured: 500 sentences, 27.7x faster than CPU.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Batch & embeddings

Zero-shot text classification on GPU

Classify texts into arbitrary labels with BART-large-MNLI (zero-shot, no training) on RTX-A4500, fanned out with .map(). Measured: ~104x faster than CPU.

  • GPU
  • Batch / map
  • Image DSL
  • Volume
Web apps & integrations

Async jobs: submit now, get the result later

Kick off a long job on a deployed endpoint without holding the HTTP connection: spawn() returns a call_id immediately, and you poll for the result whenever you want, even from another process.

  • Async job
  • Web endpoint
Web apps & integrations

Fan out hundreds of async jobs, gather the results

spawn() is non-blocking, so you can fire a whole batch of jobs at once and collect them by handle. The autoscaler adds replicas to chew through the queue in parallel.

  • Async job
  • Web endpoint
Web apps & integrations

Completion webhooks: get notified when a job finishes

Instead of polling, register a webhook: AI Fabric POSTs your URL the moment the job completes, with an HMAC signature you verify. The receiver is another AI Fabric endpoint.

  • Async job
  • Web endpoint
  • Secret

Contact the team

Loading the form…