Developer docs / AI Fabric tutorials
Batch embeddings with .map()
- GPU
- Batch / map
- Image DSL
Embedding a handful of documents is trivial: you load a model and call .encode(). The trouble starts at scale: ten thousand documents on a single GPU is a slow, serial grind, and most of that GPU sits idle waiting on one batch at a time. The fix is to fan the work out: split the corpus into chunks and embed them on many GPUs at once.
In this tutorial you'll embed 10,000 documents in parallel with AI Fabric's .map(). You write an ordinary function that embeds one chunk; AI Fabric runs it on a fresh GPU container per chunk, all at the same time, and hands the results back in input order so you can stitch them straight back onto your document list.
Before you start
You need the AI Fabric CLI installed and authenticated. Run blade setup once. There's nothing to build: we use fabric/torch, a catalog image that already ships PyTorch and sentence-transformers. The embedding function requests a single RTX A4500.
Create the app
Every AI Fabric program starts with an App. Create a file called batch_embeddings.py and declare it:
import bladekit
app = bladekit.app("batch-embeddings")Write the per-chunk embedder
The unit of parallelism is one chunk of texts. Write a function that takes a list of strings, loads the all-MiniLM-L6-v2 model onto the GPU, and returns the vectors. Request a single RTX A4500 and the fabric/torch image so there's nothing to build:
# fabric/torch catalog image ships PyTorch + sentence-transformers (no build).
# This function runs on one GPU container per batch chunk.
@app.function(gpu="RTX-A4500", image="fabric/torch", timeout=900)
def embed_chunk(texts: list[str]) -> list[list[float]]:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cuda")
vecs = model.encode(texts, batch_size=256, show_progress_bar=False)
return vecs.tolist()The function returns plain lists of floats rather than a NumPy array, so that the vectors can be serialized back to your laptop. AI Fabric runs many copies of this function at once, each on its own GPU, so the model load is paid in parallel across the fleet.
Chunk the corpus
A local_entrypoint is the part that runs on your machine. Here we simulate 10,000 documents and slice them into chunks of 500. Each chunk becomes one parallel GPU task:
@app.local_entrypoint()
def main():
# Simulate 10 000 documents in chunks of 500
docs = [f"Document number {i}: GPUs accelerate machine learning." for i in range(10_000)]
chunks = [docs[i : i + 500] for i in range(0, len(docs), 500)] Fan out with .map()
Calling embed_chunk.map(chunks) ships every chunk to its own GPU container and runs them concurrently. Passing return_exceptions=True means a single failed chunk surfaces as an exception in the results stream instead of cancelling the whole job, so you can log it and keep the rest:
all_vecs: list[list[float]] = []
for vecs in embed_chunk.map(chunks, return_exceptions=True):
if isinstance(vecs, Exception):
print(f"Chunk failed: {vecs}")
else:
all_vecs.extend(vecs)
print(f"Embedded {len(all_vecs)} documents, dim={len(all_vecs[0])}")Because .map() preserves input order, extending all_vecs chunk by chunk reconstructs the vectors in exactly the order of your original docs list, with no bookkeeping needed.
Run it
Run the program with the AI Fabric CLI:
blade run batch_embeddings.pyAI Fabric launches up to 20 GPU containers in parallel (one per chunk), each embedding 500 documents. When they finish you'll see the total count and the embedding dimension printed. For even more parallelism, split into smaller chunks (e.g. 100 docs) so more containers run at once.
Take it for a spin
Everything above in one file, ready to run.
import bladekit
app = bladekit.app("batch-embeddings")
# fabric/torch catalog image ships PyTorch + sentence-transformers, so no build.
# This function runs on one GPU container per batch chunk.
@app.function(gpu="RTX-A4500", image="fabric/torch", timeout=900)
def embed_chunk(texts: list[str]) -> list[list[float]]:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cuda")
vecs = model.encode(texts, batch_size=256, show_progress_bar=False)
return vecs.tolist()
@app.local_entrypoint()
def main():
# Simulate 10 000 documents in chunks of 500
docs = [f"Document number {i}: GPUs accelerate machine learning." for i in range(10_000)]
chunks = [docs[i : i + 500] for i in range(0, len(docs), 500)]
all_vecs: list[list[float]] = []
for vecs in embed_chunk.map(chunks, return_exceptions=True):
if isinstance(vecs, Exception):
print(f"Chunk failed: {vecs}")
else:
all_vecs.extend(vecs)
print(f"Embedded {len(all_vecs)} documents, dim={len(all_vecs[0])}")Concepts used: Cold start, Run, GPU