Developer docs / AI Fabric tutorials

Transcribe audio, then digest it with an LLM

  • GPU
  • Image DSL

Raw audio is hard to work with: you can't search it, skim it, or act on it. The fix is a small pipeline: first turn the speech into text, then turn that text into something you can actually use. In this tutorial you'll build exactly that, in two steps: Whisper transcribes a short audio clip, then a light LLM (Qwen 2.5 3B) digests the transcript into a few bullet points and a list of action items.

Both steps run on the same image and chain together with .remote() calls: the output of the transcriber becomes the input to the digester. Building on the ready fabric/torch CUDA base means you only add ffmpeg and Whisper on top: a quick layer, not a multi-gigabyte torch rebuild.

Before you start

You need the AI Fabric CLI installed and authenticated: run blade setup once. Each function asks for a single RTX A4500. Use Python 3.12 locally so it matches the catalog image (fabric/torch is built on Python 3.12), otherwise the run fails with PYTHON_VERSION_MISMATCH. You don't need to supply any audio, since the script fetches a short public clip itself, so it runs as-is.

Create the app and build the image

Create whisper_digest.py. Declare the App, then start from the fabric/torch base. CUDA, PyTorch and transformers are already baked in, so you only layer on ffmpeg (to decode audio), the openai-whisper package, and datasets/soundfile (so the script can fetch its own sample clip):

whisper_digest.py
import bladekit

app = bladekit.app("whisper-digest")

# Start from the ready fabric/torch base. CUDA + PyTorch + transformers are
# already baked in, so we only add ffmpeg + Whisper (+ datasets/soundfile so the
# script can fetch a tiny sample clip itself). A quick layer, not a multi-GB
# torch rebuild.
image = (
    bladekit.Image.from_catalog("fabric/torch")
    .apt_install("ffmpeg")
    .pip_install("openai-whisper", "datasets", "soundfile")
)

Fetch a sample clip (so it runs as-is)

A tutorial should run the moment you paste it, with no files to hunt down. So the first function grabs one short clip from the tiny public librispeech_asr_dummy dataset and hands it back as WAV bytes. The one subtlety: datasets v5 decodes audio through torchcodec by default, whose native library doesn't load on the fabric/torch base, so you cast the column with decode=False and let soundfile read the raw FLAC instead:

Python
@app.function(image=image, timeout=300)
def sample_audio() -> bytes:
    """Fetch one short, stable public clip and return it as WAV bytes so the
    tutorial is self-contained (no audio file to supply)."""
    import io, soundfile as sf
    from datasets import load_dataset, Audio

    ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
    # decode=False avoids datasets' torchcodec backend; soundfile reads the FLAC.
    ds = ds.cast_column("audio", Audio(decode=False))
    arr, sr = sf.read(io.BytesIO(ds[0]["audio"]["bytes"]))
    buf = io.BytesIO()
    sf.write(buf, arr, sr, format="WAV")
    return buf.getvalue()

Transcribe the audio (part 1)

The transcribe function takes raw audio bytes, writes them to a temp file (Whisper reads from a path), and runs the base model. The weights download on the first call and then stay warm in the container. base is fast and accurate for short clips; bump it to small or medium for harder audio:

Python
@app.function(gpu="RTX-A4500", image=image, timeout=300)
def transcribe(audio_bytes: bytes) -> str:
    import tempfile
    import whisper

    # "base" is fast and accurate for short clips; bump to "small"/"medium" for
    # harder audio. Weights download on the first call, then stay warm.
    model = whisper.load_model("base")
    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
        f.write(audio_bytes)
        path = f.name
    return model.transcribe(path)["text"].strip()

Digest the transcript with an LLM (part 2)

The second function loads Qwen 2.5 3B and turns the transcript into a digest. The _llm global keeps the model warm: the first call loads the model onto the GPU, and every later call on the same container reuses it. A system message tells the model to produce three bullet points plus any action items:

Python
_llm = None  # cached across calls in the same warm container


@app.function(gpu="RTX-A4500", image=image, timeout=600)
def digest(transcript: str) -> str:
    global _llm
    import torch
    from transformers import AutoModelForCausalLM, AutoTokenizer

    if _llm is None:
        name = "Qwen/Qwen2.5-3B-Instruct"
        tok = AutoTokenizer.from_pretrained(name)
        model = AutoModelForCausalLM.from_pretrained(
            name, torch_dtype=torch.float16
        ).to("cuda")
        _llm = (tok, model)

    tok, model = _llm
    messages = [
        {"role": "system", "content": "Summarise the transcript as 3 short bullet points, then list any action items."},
        {"role": "user", "content": transcript},
    ]
    chat = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tok(chat, return_tensors="pt").to("cuda")
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
    n_in = inputs["input_ids"].shape[1]
    return tok.decode(out[0][n_in:], skip_special_tokens=True).strip()

Chain the two steps together

The local_entrypoint runs on your machine and wires the three functions together. Call sample_audio.remote() to get the clip, pass it to transcribe.remote(...), then feed the resulting text straight into digest.remote(...). Each .remote() call ships its argument to a container, runs the function there, and returns the result:

Python
@app.local_entrypoint()
def main():
    # Self-contained: fetch a tiny public sample clip, transcribe it, digest it.
    audio = sample_audio.remote()           # tiny LibriSpeech clip as WAV bytes

    transcript = transcribe.remote(audio)   # part 1: transcribe
    print("Transcript:\n", transcript, "\n")

    summary = digest.remote(transcript)     # part 2: digest with the LLM
    print("Digest:\n", summary)

Run it

Run the program with the AI Fabric CLI:

Terminal
blade run whisper_digest.py

Part 1 runs Whisper base and prints the transcript; part 2 loads Qwen 3B and prints a three-bullet summary with action items. Both functions share one image (the fabric/torch base plus ffmpeg and Whisper), and the heavy base is reused, so the build stays quick. The model weights cache in their warm containers, so a second run skips straight to inference.

Take it for a spin

Everything above in one file, ready to run.

whisper_digest.py
# Save this file as whisper_digest.py. Do NOT name it whisper.py, or the
# "import whisper" below imports THIS file instead of the openai-whisper package
# (you'd get: AttributeError: module 'whisper' has no attribute 'load_model').
import bladekit

app = bladekit.app("whisper-digest")

# Start from the ready fabric/torch base: CUDA + PyTorch + transformers are
# already baked in, so we only add ffmpeg + Whisper (+ datasets so the script can
# fetch a tiny sample clip itself). A quick layer, not a multi-GB torch rebuild.
image = (
    bladekit.Image.from_catalog("fabric/torch")
    .apt_install("ffmpeg")
    .pip_install("openai-whisper", "datasets")
)


@app.function(image=image, timeout=300)
def sample_audio() -> bytes:
    """Fetch one short, stable public clip and return it as MP3 bytes, so the
    tutorial is self-contained (no audio file to supply) AND exercises the mp3
    path by default. ffmpeg (in the image) reads the dataset's FLAC and encodes
    a real .mp3; whisper reads it back the same way it'd read your own uploads."""
    import subprocess
    from datasets import load_dataset, Audio

    ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
    # decode=False hands us the raw file bytes (avoids datasets' torchcodec backend).
    ds = ds.cast_column("audio", Audio(decode=False))
    flac_bytes = ds[0]["audio"]["bytes"]
    proc = subprocess.run(
        ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0", "-f", "mp3", "pipe:1"],
        input=flac_bytes, capture_output=True, check=True,
    )
    return proc.stdout  # MP3 bytes


@app.function(gpu="RTX-A4500", image=image, timeout=300)
def transcribe(audio_bytes: bytes) -> str:
    import tempfile
    import whisper

    # "base" is fast and accurate for short clips; bump to "small"/"medium" for
    # harder audio. Weights download on the first call, then stay warm.
    model = whisper.load_model("base")
    # Whisper decodes via ffmpeg, which sniffs the CONTENT: mp3, m4a, wav, flac,
    # ogg all just work, so we don't force a suffix on the temp file.
    with tempfile.NamedTemporaryFile(delete=False) as f:
        f.write(audio_bytes)
        path = f.name
    return model.transcribe(path)["text"].strip()


_llm = None  # cached across calls in the same warm container


@app.function(gpu="RTX-A4500", image=image, timeout=600)
def digest(transcript: str) -> str:
    global _llm
    import torch
    from transformers import AutoModelForCausalLM, AutoTokenizer

    if _llm is None:
        name = "Qwen/Qwen2.5-3B-Instruct"
        tok = AutoTokenizer.from_pretrained(name)
        model = AutoModelForCausalLM.from_pretrained(
            name, torch_dtype=torch.float16
        ).to("cuda")
        _llm = (tok, model)

    tok, model = _llm
    messages = [
        {"role": "system", "content": "Summarise the transcript as 3 short bullet points, then list any action items."},
        {"role": "user", "content": transcript},
    ]
    chat = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tok(chat, return_tensors="pt").to("cuda")
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
    n_in = inputs["input_ids"].shape[1]
    return tok.decode(out[0][n_in:], skip_special_tokens=True).strip()


@app.local_entrypoint()
def main():
    # Self-contained: fetch a tiny public sample clip, transcribe it, digest it.
    audio = sample_audio.remote()           # tiny LibriSpeech clip as MP3 bytes

    transcript = transcribe.remote(audio)   # part 1: transcribe
    print("Transcript:\n", transcript, "\n")

    summary = digest.remote(transcript)     # part 2: digest with the LLM
    print("Digest:\n", summary)

Concepts used: Cold start, Run, GPU

Contact the team

Loading the form…