Developer docs / AI Fabric tutorials

Batch image captioning on GPU

  • GPU
  • Batch / map
  • Image DSL
  • Volume

Captioning a folder of images is the classic embarrassingly parallel job: every image is independent, so there's no reason to caption them one at a time. The slow part is loading a vision-language model onto a GPU, and once it's loaded, a single caption takes a fraction of a second. The win is to run many GPUs at once, each captioning its own image.

In this tutorial you'll caption a batch of images with BLIP-large on RTX-A4500 GPUs, fanned out with .map() so AI Fabric launches one GPU container per image, all running in parallel. The images are fetched inside the containers, so there are no local files to wrangle, and per-image timing is printed so you can watch the GPU caption an image in under a second.

Before you start

You need the AI Fabric CLI installed and authenticated: run blade setup once. There's nothing to build locally: this job runs on fabric/torch, a catalog image that already ships CUDA, PyTorch and Python 3.12, and each image is captioned on a single RTX-A4500.

Create the app and build the image

Create a file called caption_batch.py. Start from the fabric/torch catalog base and layer on the few extras BLIP needs. We pin transformers to a known-good version so a future API change can't break BLIP's built-in classes, and point HF_HOME at a Volume so the weights download only once:

caption_batch.py
import time

import bladekit

app = bladekit.app("batch-captioning")

# Cache HF model weights across runs (downloaded once, reused forever).
hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install("transformers==4.46.3", "pillow", "numpy")
    .env({"HF_HOME": "/hf-cache"})
)

Pick a model and some images

BLIP-large is ~1.9 GB and public on the Hugging Face Hub, so it fits easily in the A4500's 20 GB. For sample inputs we use a handful of Lorem Picsum URLs. Each image is fetched inside the container, so you never copy files up to the fleet:

Python
MODEL_ID = "Salesforce/blip-image-captioning-large"  # ~1.9 GB, public on the hub

# A few small sample images from Lorem Picsum (reliable, any size, no hotlink
# limits). Each is fetched inside the container, so there are NO local files.
SAMPLE_URLS = [
    "https://picsum.photos/seed/forge-cat/512",
    "https://picsum.photos/seed/forge-city/512",
    "https://picsum.photos/seed/forge-beach/512",
    "https://picsum.photos/seed/forge-forest/512",
    "https://picsum.photos/seed/forge-market/512",
]

# Loaded once per container, reused across every call that lands on it.
_processor = None
_model = None

Write the caption function

The caption function takes a single URL and runs on one GPU. Request an RTX-A4500, the image you built, and mount the Volume at /hf-cache. The _model / _processor globals cache the weights per container: the first call on a container loads the weights, and every later call routed to that same hot container reuses them with no reload:

Python
@app.function(
    gpu="RTX-A4500",
    image=image,
    timeout=900,
    volumes={"/hf-cache": hf_cache},
)
def caption(url: str) -> dict:
    """Download one image, caption it on the GPU, return the caption + timing."""
    global _processor, _model
    import io
    import time as _t
    import urllib.request

    import torch
    from PIL import Image as PILImage
    from transformers import BlipForConditionalGeneration, BlipProcessor

    device = "cuda" if torch.cuda.is_available() else "cpu"

    # Load processor + model once per container; weights auto-cache to /hf-cache.
    # The next call routed to this hot container reuses them (no reload).
    if _model is None:
        t_load = _t.perf_counter()
        _processor = BlipProcessor.from_pretrained(MODEL_ID)
        _model = (
            BlipForConditionalGeneration.from_pretrained(MODEL_ID)
            .to(device)
            .eval()
        )
        print(f"[remote] model loaded on {device} in {_t.perf_counter() - t_load:.1f}s")

With the model resident, the rest is one image's worth of work: fetch the bytes, decode them with PIL, turn the picture into pixel_values, generate the caption on the GPU, and time only the inference. Returning a dict means each result carries its caption alongside the timing and image size:

Python
    # Fetch the sample image into memory (no local files required).
    req = urllib.request.Request(url, headers={"User-Agent": "fabric-tutorial/1.0"})
    with urllib.request.urlopen(req, timeout=30) as resp:
        raw = resp.read()
    img = PILImage.open(io.BytesIO(raw)).convert("RGB")

    # The BLIP processor turns the PIL image into pixel_values (PIL + numpy under
    # the hood, no torchvision). Move the tensors to the GPU.
    inputs = _processor(images=img, return_tensors="pt").to(device)

    # Generate the caption on the GPU and time just the inference.
    t0 = _t.perf_counter()
    with torch.no_grad():
        out_ids = _model.generate(**inputs, max_new_tokens=30)
    if device == "cuda":
        torch.cuda.synchronize()
    gpu_seconds = _t.perf_counter() - t0

    text = _processor.decode(out_ids[0], skip_special_tokens=True).strip()

    return {
        "url": url,
        "caption": text,
        "in_size": list(img.size),  # (W, H)
        "gpu_seconds": round(gpu_seconds, 3),
        "device": device,
    }

Fan out with .map()

The local_entrypoint runs on your laptop and kicks off the batch. caption.map(SAMPLE_URLS) is the whole fan-out: AI Fabric sends each URL to its own GPU container and runs them all at once. return_exceptions=True keeps one bad URL from cancelling the rest, and results come back in input order so you can zip them straight back to SAMPLE_URLS:

Python
@app.local_entrypoint()
def main():
    print(
        f"Captioning {len(SAMPLE_URLS)} images with BLIP, "
        f"fanning out across the fleet...\n"
    )

    t0 = time.perf_counter()
    # One GPU container per image, all in parallel. return_exceptions keeps a
    # single bad URL from cancelling the rest of the batch; results come back in
    # input order so we can zip them to SAMPLE_URLS.
    results = list(caption.map(SAMPLE_URLS, return_exceptions=True))
    wall = time.perf_counter() - t0

The rest of main prints each caption with its timing and a small summary. The headline number is GPU seconds per image versus the wall-clock time. Because every image ran on its own container in parallel, the wall-clock barely grows as you add more images:

Python
    ok = []
    for url, res in zip(SAMPLE_URLS, results):
        name = url.rsplit("/", 1)[-2]  # the seed name in the URL
        if isinstance(res, Exception):
            print(f"  FAILED {name}: {res}")
            continue
        ok.append(res)
        print(
            f'  {name} ({tuple(res["in_size"])}, {res["gpu_seconds"]}s on '
            f'{res["device"]}): "{res["caption"]}"'
        )

Run it

Run the program with the AI Fabric CLI:

Terminal
blade run caption_batch.py

The first run downloads the BLIP weights (~1.9 GB) into the hf-cache Volume. That's the one-time cost. Later runs skip the download entirely. When it finishes you'll see one line per image with its caption and the GPU time it actually used:

Output
  forge-cat ((512, 512), 0.34s on cuda): "a cat sitting on a window sill"

--- timing ---
  images captioned: 5/5
  GPU inference:    0.350s/image (sum 1.75s)
  wall-clock:       ... (includes parallel cold start + download)

Each image ran on its own GPU container in parallel, and AI Fabric bills per started minute, so you pay a GPU-minute or two per container, not for an always-on server. To caption your own pictures, point SAMPLE_URLS at your URLs, or read the images from a Volume or object store instead.

Take it for a spin

Everything above in one file, ready to run.

caption_batch.py
"""Batch image captioning on the GPU fleet, fanned out with .map().

Fetch a handful of small public sample images, caption each one on an
RTX-A4500 with a transformers vision-language model (BLIP), and fan the batch
out across the fleet with ``.map()``, one GPU container per image, all running
at once. The function prints each caption plus per-image timing, so you can see
the GPU caption an image in a fraction of a second.

    blade run batch_captioning.py

The BLIP weights (~1.9 GB) download once from the Hugging Face Hub and are
cached on a Volume, so later runs skip the download. The model fits easily in
the A4500's 20 GB.
"""

import time

import bladekit

app = bladekit.app("batch-captioning")

# Cache HF model weights across runs (downloaded once, reused forever).
hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

# fabric/torch = CUDA + PyTorch + Python 3.12 (AI Fabric catalog, no build of its own).
# BLIP ships in 🤗 transformers: pure pip, no git URLs, no compilation. We pin
# transformers to a known-good version so a future API change can't break the
# built-in BLIP classes. BLIP's image processor uses PIL + numpy, NOT
# torchvision, so the lean fabric/torch base (which has no torchvision) is enough.
image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install("transformers==4.46.3", "pillow", "numpy")
    .env({"HF_HOME": "/hf-cache"})
)

MODEL_ID = "Salesforce/blip-image-captioning-large"  # ~1.9 GB, public on the hub

# A few small sample images from Lorem Picsum (reliable, any size, no hotlink
# limits). Each is fetched inside the container, so there are NO local files.
SAMPLE_URLS = [
    "https://picsum.photos/seed/forge-cat/512",
    "https://picsum.photos/seed/forge-city/512",
    "https://picsum.photos/seed/forge-beach/512",
    "https://picsum.photos/seed/forge-forest/512",
    "https://picsum.photos/seed/forge-market/512",
]

# Loaded once per container, reused across every call that lands on it.
_processor = None
_model = None


@app.function(
    gpu="RTX-A4500",
    image=image,
    timeout=900,
    volumes={"/hf-cache": hf_cache},
)
def caption(url: str) -> dict:
    """Download one image, caption it on the GPU, return the caption + timing."""
    global _processor, _model
    import io
    import time as _t
    import urllib.request

    import torch
    from PIL import Image as PILImage
    from transformers import BlipForConditionalGeneration, BlipProcessor

    device = "cuda" if torch.cuda.is_available() else "cpu"

    # Load processor + model once per container; weights auto-cache to /hf-cache.
    # The next call routed to this hot container reuses them (no reload).
    if _model is None:
        t_load = _t.perf_counter()
        _processor = BlipProcessor.from_pretrained(MODEL_ID)
        _model = (
            BlipForConditionalGeneration.from_pretrained(MODEL_ID)
            .to(device)
            .eval()
        )
        print(f"[remote] model loaded on {device} in {_t.perf_counter() - t_load:.1f}s")

    # Fetch the sample image into memory (no local files required).
    req = urllib.request.Request(url, headers={"User-Agent": "fabric-tutorial/1.0"})
    with urllib.request.urlopen(req, timeout=30) as resp:
        raw = resp.read()
    img = PILImage.open(io.BytesIO(raw)).convert("RGB")

    # The BLIP processor turns the PIL image into pixel_values (PIL + numpy under
    # the hood, no torchvision). Move the tensors to the GPU.
    inputs = _processor(images=img, return_tensors="pt").to(device)

    # Generate the caption on the GPU and time just the inference.
    t0 = _t.perf_counter()
    with torch.no_grad():
        out_ids = _model.generate(**inputs, max_new_tokens=30)
    if device == "cuda":
        torch.cuda.synchronize()
    gpu_seconds = _t.perf_counter() - t0

    text = _processor.decode(out_ids[0], skip_special_tokens=True).strip()

    return {
        "url": url,
        "caption": text,
        "in_size": list(img.size),  # (W, H)
        "gpu_seconds": round(gpu_seconds, 3),
        "device": device,
    }


@app.local_entrypoint()
def main():
    print(
        f"Captioning {len(SAMPLE_URLS)} images with BLIP, "
        f"fanning out across the fleet...\n"
    )

    t0 = time.perf_counter()
    # One GPU container per image, all in parallel. return_exceptions keeps a
    # single bad URL from cancelling the rest of the batch; results come back in
    # input order so we can zip them to SAMPLE_URLS.
    results = list(caption.map(SAMPLE_URLS, return_exceptions=True))
    wall = time.perf_counter() - t0

    ok = []
    for url, res in zip(SAMPLE_URLS, results):
        name = url.rsplit("/", 1)[-2]  # the seed name in the URL
        if isinstance(res, Exception):
            print(f"  FAILED {name}: {res}")
            continue
        ok.append(res)
        print(
            f'  {name} ({tuple(res["in_size"])}, {res["gpu_seconds"]}s on '
            f'{res["device"]}): "{res["caption"]}"'
        )

    if not ok:
        print("\nNo captions produced. Check the run logs above.")
        return

    total_gpu = sum(r["gpu_seconds"] for r in ok)
    avg_gpu = total_gpu / len(ok)
    print("\n--- timing ---")
    print(f"  images captioned: {len(ok)}/{len(SAMPLE_URLS)}")
    print(f"  GPU inference:    {avg_gpu:.3f}s/image (sum {total_gpu:.2f}s)")
    print(f"  wall-clock:       {wall:.1f}s (includes parallel cold start + download)")
    print(
        "\nEach image ran on its own GPU container in parallel, and AI Fabric bills "
        "per started minute, so you pay a GPU-minute or two per container, "
        "not for an always-on server."
    )

Concepts used: GPU, Run, Volume

Contact the team

Loading the form…