Developer docs / AI Fabric tutorials
Transcribe audio, then digest it with an LLM
- GPU
- Image DSL
Raw audio is hard to work with: you can't search it, skim it, or act on it. The fix is a small pipeline: first turn the speech into text, then turn that text into something you can actually use. In this tutorial you'll build exactly that, in two steps: Whisper transcribes a short audio clip, then a light LLM (Qwen 2.5 3B) digests the transcript into a few bullet points and a list of action items.
Both steps run on the same image and chain together with .remote() calls: the output of the transcriber becomes the input to the digester. Building on the ready fabric/torch CUDA base means you only add ffmpeg and Whisper on top: a quick layer, not a multi-gigabyte torch rebuild.
Before you start
You need the AI Fabric CLI installed and authenticated: run blade setup once. Each function asks for a single RTX A4500. Use Python 3.12 locally so it matches the catalog image (fabric/torch is built on Python 3.12), otherwise the run fails with PYTHON_VERSION_MISMATCH. You don't need to supply any audio, since the script fetches a short public clip itself, so it runs as-is.
Create the app and build the image
Create whisper_digest.py. Declare the App, then start from the fabric/torch base. CUDA, PyTorch and transformers are already baked in, so you only layer on ffmpeg (to decode audio), the openai-whisper package, and datasets/soundfile (so the script can fetch its own sample clip):
import bladekit
app = bladekit.app("whisper-digest")
# Start from the ready fabric/torch base. CUDA + PyTorch + transformers are
# already baked in, so we only add ffmpeg + Whisper (+ datasets/soundfile so the
# script can fetch a tiny sample clip itself). A quick layer, not a multi-GB
# torch rebuild.
image = (
bladekit.Image.from_catalog("fabric/torch")
.apt_install("ffmpeg")
.pip_install("openai-whisper", "datasets", "soundfile")
)Fetch a sample clip (so it runs as-is)
A tutorial should run the moment you paste it, with no files to hunt down. So the first function grabs one short clip from the tiny public librispeech_asr_dummy dataset and hands it back as WAV bytes. The one subtlety: datasets v5 decodes audio through torchcodec by default, whose native library doesn't load on the fabric/torch base, so you cast the column with decode=False and let soundfile read the raw FLAC instead:
@app.function(image=image, timeout=300)
def sample_audio() -> bytes:
"""Fetch one short, stable public clip and return it as WAV bytes so the
tutorial is self-contained (no audio file to supply)."""
import io, soundfile as sf
from datasets import load_dataset, Audio
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
# decode=False avoids datasets' torchcodec backend; soundfile reads the FLAC.
ds = ds.cast_column("audio", Audio(decode=False))
arr, sr = sf.read(io.BytesIO(ds[0]["audio"]["bytes"]))
buf = io.BytesIO()
sf.write(buf, arr, sr, format="WAV")
return buf.getvalue()Transcribe the audio (part 1)
The transcribe function takes raw audio bytes, writes them to a temp file (Whisper reads from a path), and runs the base model. The weights download on the first call and then stay warm in the container. base is fast and accurate for short clips; bump it to small or medium for harder audio:
@app.function(gpu="RTX-A4500", image=image, timeout=300)
def transcribe(audio_bytes: bytes) -> str:
import tempfile
import whisper
# "base" is fast and accurate for short clips; bump to "small"/"medium" for
# harder audio. Weights download on the first call, then stay warm.
model = whisper.load_model("base")
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
f.write(audio_bytes)
path = f.name
return model.transcribe(path)["text"].strip()Digest the transcript with an LLM (part 2)
The second function loads Qwen 2.5 3B and turns the transcript into a digest. The _llm global keeps the model warm: the first call loads the model onto the GPU, and every later call on the same container reuses it. A system message tells the model to produce three bullet points plus any action items:
_llm = None # cached across calls in the same warm container
@app.function(gpu="RTX-A4500", image=image, timeout=600)
def digest(transcript: str) -> str:
global _llm
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
if _llm is None:
name = "Qwen/Qwen2.5-3B-Instruct"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(
name, torch_dtype=torch.float16
).to("cuda")
_llm = (tok, model)
tok, model = _llm
messages = [
{"role": "system", "content": "Summarise the transcript as 3 short bullet points, then list any action items."},
{"role": "user", "content": transcript},
]
chat = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(chat, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
n_in = inputs["input_ids"].shape[1]
return tok.decode(out[0][n_in:], skip_special_tokens=True).strip()Chain the two steps together
The local_entrypoint runs on your machine and wires the three functions together. Call sample_audio.remote() to get the clip, pass it to transcribe.remote(...), then feed the resulting text straight into digest.remote(...). Each .remote() call ships its argument to a container, runs the function there, and returns the result:
@app.local_entrypoint()
def main():
# Self-contained: fetch a tiny public sample clip, transcribe it, digest it.
audio = sample_audio.remote() # tiny LibriSpeech clip as WAV bytes
transcript = transcribe.remote(audio) # part 1: transcribe
print("Transcript:\n", transcript, "\n")
summary = digest.remote(transcript) # part 2: digest with the LLM
print("Digest:\n", summary)Run it
Run the program with the AI Fabric CLI:
blade run whisper_digest.pyPart 1 runs Whisper base and prints the transcript; part 2 loads Qwen 3B and prints a three-bullet summary with action items. Both functions share one image (the fabric/torch base plus ffmpeg and Whisper), and the heavy base is reused, so the build stays quick. The model weights cache in their warm containers, so a second run skips straight to inference.
Take it for a spin
Everything above in one file, ready to run.
# Save this file as whisper_digest.py. Do NOT name it whisper.py, or the
# "import whisper" below imports THIS file instead of the openai-whisper package
# (you'd get: AttributeError: module 'whisper' has no attribute 'load_model').
import bladekit
app = bladekit.app("whisper-digest")
# Start from the ready fabric/torch base: CUDA + PyTorch + transformers are
# already baked in, so we only add ffmpeg + Whisper (+ datasets so the script can
# fetch a tiny sample clip itself). A quick layer, not a multi-GB torch rebuild.
image = (
bladekit.Image.from_catalog("fabric/torch")
.apt_install("ffmpeg")
.pip_install("openai-whisper", "datasets")
)
@app.function(image=image, timeout=300)
def sample_audio() -> bytes:
"""Fetch one short, stable public clip and return it as MP3 bytes, so the
tutorial is self-contained (no audio file to supply) AND exercises the mp3
path by default. ffmpeg (in the image) reads the dataset's FLAC and encodes
a real .mp3; whisper reads it back the same way it'd read your own uploads."""
import subprocess
from datasets import load_dataset, Audio
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
# decode=False hands us the raw file bytes (avoids datasets' torchcodec backend).
ds = ds.cast_column("audio", Audio(decode=False))
flac_bytes = ds[0]["audio"]["bytes"]
proc = subprocess.run(
["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0", "-f", "mp3", "pipe:1"],
input=flac_bytes, capture_output=True, check=True,
)
return proc.stdout # MP3 bytes
@app.function(gpu="RTX-A4500", image=image, timeout=300)
def transcribe(audio_bytes: bytes) -> str:
import tempfile
import whisper
# "base" is fast and accurate for short clips; bump to "small"/"medium" for
# harder audio. Weights download on the first call, then stay warm.
model = whisper.load_model("base")
# Whisper decodes via ffmpeg, which sniffs the CONTENT: mp3, m4a, wav, flac,
# ogg all just work, so we don't force a suffix on the temp file.
with tempfile.NamedTemporaryFile(delete=False) as f:
f.write(audio_bytes)
path = f.name
return model.transcribe(path)["text"].strip()
_llm = None # cached across calls in the same warm container
@app.function(gpu="RTX-A4500", image=image, timeout=600)
def digest(transcript: str) -> str:
global _llm
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
if _llm is None:
name = "Qwen/Qwen2.5-3B-Instruct"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(
name, torch_dtype=torch.float16
).to("cuda")
_llm = (tok, model)
tok, model = _llm
messages = [
{"role": "system", "content": "Summarise the transcript as 3 short bullet points, then list any action items."},
{"role": "user", "content": transcript},
]
chat = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(chat, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
n_in = inputs["input_ids"].shape[1]
return tok.decode(out[0][n_in:], skip_special_tokens=True).strip()
@app.local_entrypoint()
def main():
# Self-contained: fetch a tiny public sample clip, transcribe it, digest it.
audio = sample_audio.remote() # tiny LibriSpeech clip as MP3 bytes
transcript = transcribe.remote(audio) # part 1: transcribe
print("Transcript:\n", transcript, "\n")
summary = digest.remote(transcript) # part 2: digest with the LLM
print("Digest:\n", summary)Concepts used: Cold start, Run, GPU