Developer docs / AI Fabric tutorials

Embed a whole dataset on the GPU fleet

  • GPU
  • Batch / map
  • Image DSL

Embedding a dataset is embarrassingly parallel: every document is independent, so there's no reason to wait for one to finish before starting the next. On a GPU the work is fast; spread across a fleet of GPUs it's nearly instant. And because AI Fabric bills per started minute, a job that finishes in seconds costs you a GPU-minute per container, not the price of a GPU you rented by the hour.

In this tutorial you'll embed a few hundred sentences with BGE-small on RTX A4500s, fanned out with .map(). Each worker also times a CPU baseline on the same data, so the speedup the script prints (~25x here) is measured, not asserted. There's nothing to download or set up, since the dataset is built in-process.

Before you start

You need the AI Fabric CLI installed and authenticated. Run blade setup once. The embedding function runs on a single RTX A4500, and we build the image on top of fabric/torch (the catalog base: CUDA + PyTorch + Python 3.12) by adding sentence-transformers, with no Dockerfile and no manual setup.

Create the app and image

Create dataset_embeddings.py. Declare the app and define the image by extending the catalog base with the one library we need. We also pin the model id up front: BGE-small is 384-dimensional, ~130 MB, and public on the HF hub:

dataset_embeddings.py
import bladekit

app = bladekit.app("dataset-embeddings")

# fabric/torch is the AI Fabric catalog base (CUDA + PyTorch + Python 3.12). We add
# sentence-transformers on top; no Dockerfile, no manual setup.
image = bladekit.Image.from_catalog("fabric/torch").pip_install(
    "sentence-transformers==3.0.1"
)

MODEL_ID = "BAAI/bge-small-en-v1.5"  # 384-dim, ~130 MB, public on the HF hub

Build a self-contained dataset

So the tutorial runs with zero setup, generate the corpus in-process by combining a handful of subjects and predicates into a few hundred varied sentences. In real use you'd replace this with any list of strings, and the docstring shows how to drop in a HuggingFace dataset slice instead:

Python
def build_dataset(n: int = 400) -> list[str]:
    """A small, self-contained text dataset: a few hundred varied sentences.

    Replace this with any list of strings, e.g. a HuggingFace dataset slice:
        from datasets import load_dataset
        rows = load_dataset("ag_news", split="train[:400]")["text"]
    """
    subjects = [
        "A serverless GPU platform", "The embedding model", "This dataset",
        "A transformer network", "The vector index", "Our retrieval pipeline",
        "The fleet scheduler", "A batch of documents", "The cold-start path",
        "Per-minute billing",
    ]
    predicates = [
        "scales out across many GPUs in parallel.",
        "turns text into dense vectors in milliseconds.",
        "is cached so the next run starts instantly.",
        "fans work out with a single .map() call.",
        "fits comfortably in 20 GB of VRAM.",
        "runs far faster on a GPU than on a CPU.",
        "returns results in the original input order.",
        "only costs you the minutes it actually runs.",
    ]
    docs = []
    i = 0
    while len(docs) < n:
        s = subjects[i % len(subjects)]
        p = predicates[(i // len(subjects)) % len(predicates)]
        docs.append(f"{s} {p} (row {len(docs)})")
        i += 1
    return docs

Embed a chunk and measure the speedup

The unit of parallelism is one chunk of sentences. The function loads BGE onto the GPU once per container, encodes the chunk, and times it. To prove the GPU is worth it, it then encodes the same chunk on the CPU and times that too, so the caller can report a measured GPU-vs-CPU ratio:

Python
@app.function(gpu="RTX-A4500", image=image, timeout=900)
def embed_chunk(texts: list[str]) -> dict:
    """Embed one chunk of sentences on the GPU. Loads the model once per container.

    Returns the vectors plus timing for both GPU and a one-shot CPU baseline so
    the caller can report a real speedup.
    """
    import time

    import torch
    from sentence_transformers import SentenceTransformer

    # Load once per container; subsequent chunks routed to this hot container
    # reuse the already-loaded model (no reload, no re-download).
    gpu_model = SentenceTransformer(MODEL_ID, device="cuda")

    # --- GPU timing (the real workload) ---
    torch.cuda.synchronize()
    t0 = time.perf_counter()
    vecs = gpu_model.encode(texts, batch_size=128, show_progress_bar=False)
    torch.cuda.synchronize()
    gpu_seconds = time.perf_counter() - t0

    # --- CPU baseline on the SAME chunk, for an honest speedup number ---
    cpu_model = SentenceTransformer(MODEL_ID, device="cpu")
    t0 = time.perf_counter()
    cpu_model.encode(texts, batch_size=128, show_progress_bar=False)
    cpu_seconds = time.perf_counter() - t0

    return {
        "vectors": vecs.tolist(),
        "n": len(texts),
        "dim": int(vecs.shape[1]),
        "gpu_seconds": gpu_seconds,
        "cpu_seconds": cpu_seconds,
    }

Fan out and tally the results

In the entrypoint, slice the dataset into chunks of 100 and fan them out with .map(): one GPU container per chunk, all in parallel. Results arrive in input order, so you can extend all_vectors straight through and sum the per-chunk timings as they come back:

Python
@app.local_entrypoint()
def main():
    docs = build_dataset(400)
    chunk_size = 100
    chunks = [docs[i : i + chunk_size] for i in range(0, len(docs), chunk_size)]
    print(f"Embedding {len(docs)} documents in {len(chunks)} chunks "
          f"across the GPU fleet...")

    all_vectors: list[list[float]] = []
    total_gpu = 0.0
    total_cpu = 0.0
    dim = None

    # .map() fans the chunks out: one GPU container per chunk, all in parallel.
    # Results arrive in input order, so we can zip them back to `docs`.
    for res in embed_chunk.map(chunks, return_exceptions=True):
        if isinstance(res, Exception):
            print(f"  chunk failed: {res}")
            continue
        all_vectors.extend(res["vectors"])
        total_gpu += res["gpu_seconds"]
        total_cpu += res["cpu_seconds"]
        dim = res["dim"]

return_exceptions=True keeps one bad chunk from sinking the whole run: a failure is yielded as an exception you can log and skip. Finally, print the totals and the measured speedup:

Python
    if not all_vectors:
        print("No vectors produced. Check the run logs above.")
        return

    print(f"\nEmbedded {len(all_vectors)} documents, dim={dim}")
    print(f"First vector preview: {all_vectors[0][:6]} ...")

    speedup = (total_cpu / total_gpu) if total_gpu > 0 else float("nan")
    print("\n--- timing (summed over all chunks) ---")
    print(f"  GPU encode time: {total_gpu:6.2f}s")
    print(f"  CPU encode time: {total_cpu:6.2f}s")
    print(f"  speedup:         {speedup:5.1f}x faster on GPU")
    print("\nAI Fabric bills each container per started minute: this whole job used a few "
          "GPU-minutes across the fleet, and nothing while idle.")

Run it

Run the program with the AI Fabric CLI:

Terminal
blade run dataset_embeddings.py

AI Fabric fans the chunks across the GPU fleet, each container embeds its slice, and the vectors come back in order. The printed speedup (~25x here) is the summed CPU time divided by the summed GPU time, a number your own run produced, not one we made up. Swap build_dataset() for any list of strings and the same pipeline embeds your real data.

Take it for a spin

Everything above in one file, ready to run.

dataset_embeddings.py
"""Dataset embeddings on the GPU fleet, fanned out with .map().

Run with:  blade run dataset_embeddings.py

What it does
------------
- Builds a small text dataset in-process (a few hundred sentences), no files
  to download, no setup. Swap in any list of strings (e.g. a HF dataset slice).
- Embeds every sentence with the BGE-small sentence-transformers model on a
  RTX-A4500 GPU. The model (~130 MB) downloads from the public HF hub at runtime
  and fits comfortably in 20 GB of VRAM.
- Splits the dataset into chunks and uses `.map()` to fan the chunks out across
  the GPU fleet. Many containers embed in parallel, results come back in order.
- Each GPU worker also embeds its first chunk on the CPU once, so the script can
  print a real measured GPU-vs-CPU speedup, not a hand-waved number.

The headline: hundreds of docs embedded in seconds on GPU vs minutes on CPU,
and AI Fabric bills each container per started minute, so the job costs a few GPU-minutes.
"""

import bladekit

app = bladekit.app("dataset-embeddings")

# fabric/torch is the AI Fabric catalog base (CUDA + PyTorch + Python 3.12). We add
# sentence-transformers on top; no Dockerfile, no manual setup.
image = bladekit.Image.from_catalog("fabric/torch").pip_install(
    "sentence-transformers==3.0.1"
)

MODEL_ID = "BAAI/bge-small-en-v1.5"  # 384-dim, ~130 MB, public on the HF hub


def build_dataset(n: int = 400) -> list[str]:
    """A small, self-contained text dataset: a few hundred varied sentences.

    Replace this with any list of strings, e.g. a HuggingFace dataset slice:
        from datasets import load_dataset
        rows = load_dataset("ag_news", split="train[:400]")["text"]
    """
    subjects = [
        "A serverless GPU platform", "The embedding model", "This dataset",
        "A transformer network", "The vector index", "Our retrieval pipeline",
        "The fleet scheduler", "A batch of documents", "The cold-start path",
        "Per-minute billing",
    ]
    predicates = [
        "scales out across many GPUs in parallel.",
        "turns text into dense vectors in milliseconds.",
        "is cached so the next run starts instantly.",
        "fans work out with a single .map() call.",
        "fits comfortably in 20 GB of VRAM.",
        "runs far faster on a GPU than on a CPU.",
        "returns results in the original input order.",
        "only costs you the minutes it actually runs.",
    ]
    docs = []
    i = 0
    while len(docs) < n:
        s = subjects[i % len(subjects)]
        p = predicates[(i // len(subjects)) % len(predicates)]
        docs.append(f"{s} {p} (row {len(docs)})")
        i += 1
    return docs


@app.function(gpu="RTX-A4500", image=image, timeout=900)
def embed_chunk(texts: list[str]) -> dict:
    """Embed one chunk of sentences on the GPU. Loads the model once per container.

    Returns the vectors plus timing for both GPU and a one-shot CPU baseline so
    the caller can report a real speedup.
    """
    import time

    import torch
    from sentence_transformers import SentenceTransformer

    # Load once per container; subsequent chunks routed to this hot container
    # reuse the already-loaded model (no reload, no re-download).
    gpu_model = SentenceTransformer(MODEL_ID, device="cuda")

    # --- GPU timing (the real workload) ---
    torch.cuda.synchronize()
    t0 = time.perf_counter()
    vecs = gpu_model.encode(texts, batch_size=128, show_progress_bar=False)
    torch.cuda.synchronize()
    gpu_seconds = time.perf_counter() - t0

    # --- CPU baseline on the SAME chunk, for an honest speedup number ---
    cpu_model = SentenceTransformer(MODEL_ID, device="cpu")
    t0 = time.perf_counter()
    cpu_model.encode(texts, batch_size=128, show_progress_bar=False)
    cpu_seconds = time.perf_counter() - t0

    return {
        "vectors": vecs.tolist(),
        "n": len(texts),
        "dim": int(vecs.shape[1]),
        "gpu_seconds": gpu_seconds,
        "cpu_seconds": cpu_seconds,
    }


@app.local_entrypoint()
def main():
    docs = build_dataset(400)
    chunk_size = 100
    chunks = [docs[i : i + chunk_size] for i in range(0, len(docs), chunk_size)]
    print(f"Embedding {len(docs)} documents in {len(chunks)} chunks "
          f"across the GPU fleet...")

    all_vectors: list[list[float]] = []
    total_gpu = 0.0
    total_cpu = 0.0
    dim = None

    # .map() fans the chunks out: one GPU container per chunk, all in parallel.
    # Results arrive in input order, so we can zip them back to `docs`.
    for res in embed_chunk.map(chunks, return_exceptions=True):
        if isinstance(res, Exception):
            print(f"  chunk failed: {res}")
            continue
        all_vectors.extend(res["vectors"])
        total_gpu += res["gpu_seconds"]
        total_cpu += res["cpu_seconds"]
        dim = res["dim"]

    if not all_vectors:
        print("No vectors produced. Check the run logs above.")
        return

    print(f"\nEmbedded {len(all_vectors)} documents, dim={dim}")
    print(f"First vector preview: {all_vectors[0][:6]} ...")

    speedup = (total_cpu / total_gpu) if total_gpu > 0 else float("nan")
    print("\n--- timing (summed over all chunks) ---")
    print(f"  GPU encode time: {total_gpu:6.2f}s")
    print(f"  CPU encode time: {total_cpu:6.2f}s")
    print(f"  speedup:         {speedup:5.1f}x faster on GPU")
    print("\nAI Fabric bills each container per started minute: this whole job used a few "
          "GPU-minutes across the fleet, and nothing while idle.")

Concepts used: GPU, Run, Cold start

Contact the team

Loading the form…