Developer docs / AI Fabric tutorials
Whisper batch transcription
- GPU
- Batch / map
- Image DSL
Transcribing one audio file on a GPU is quick. Transcribing a thousand of them one after another is a long afternoon, even though each file is independent and there's no reason they can't all run at once. The bottleneck isn't the model, it's that a single container processes one file at a time.
In this tutorial you'll transcribe a whole folder of audio with Whisper large-v3 and fan the work out across many GPU containers at once using .map(). You write a function that handles one file; AI Fabric runs it on hundreds of files in parallel and hands you the results in order. No queue, no worker pool, no orchestration code.
Before you start
You need the AI Fabric CLI installed and authenticated: run blade setup once. The function asks for a single RTX A4500, and the image builds on fabric/torch (a catalog image with CUDA and PyTorch already baked in), so there's no multi-GB build to wait on.
Create the app
Create a file called whisper_batch.py. Every AI Fabric program starts with an App, the namespace your functions live under:
import bladekit
app = bladekit.app("whisper-batch")Build the image
Whisper needs two things on top of the base CUDA stack: the openai-whisper package and ffmpeg to decode audio. Start from the fabric/torch catalog image and layer them on. PyTorch is already there, so this is a thin, fast build rather than a full torch rebuild:
image = (
bladekit.Image.from_catalog("fabric/torch")
.pip_install("torch", "openai-whisper", "ffmpeg-python")
.apt_install("ffmpeg")
)Write the per-file transcriber
The key idea: you write the function as if it only ever sees one file. It takes the raw bytes of a single clip, loads the Whisper large-v3 model, writes the bytes to a temp file (ffmpeg reads from disk), and returns the transcript text:
@app.function(gpu="RTX-A4500", image=image, timeout=600)
def transcribe(audio_bytes: bytes) -> str:
import tempfile, whisper
model = whisper.load_model("large-v3")
with tempfile.NamedTemporaryFile(suffix=".mp3", delete=False) as f:
f.write(audio_bytes)
tmp_path = f.name
result = model.transcribe(tmp_path)
return result["text"]large-v3 is the most accurate Whisper model and well worth the GPU when you're processing in bulk. The weights download on a container's first call and stay resident while it's warm, so back-to-back files on the same container skip the download.
Fan out with .map()
The local_entrypoint runs on your laptop. Glob the audio/ folder, read each file into bytes, then call transcribe.map(...). This is where the parallelism happens. AI Fabric sends each item to its own GPU container and runs them all at the same time:
@app.local_entrypoint()
def main():
import pathlib
# Each audio file is sent to a separate GPU container in parallel.
audio_files = list(pathlib.Path("audio/").glob("*.mp3"))
audio_bytes_list = [f.read_bytes() for f in audio_files]
results = list(transcribe.map(audio_bytes_list, return_exceptions=True)).map() returns results in input order, so you can zip them straight back to the files they came from. Passing return_exceptions=True keeps one bad file from sinking the batch: if one file is corrupt and raises, you get the exception as that item's result instead of losing the whole run.
Report results
Walk the zipped pairs and print each transcript (or the error, if that file failed). Because exceptions came back inline, a single bad clip prints FAILED and the rest carry on:
for path, text in zip(audio_files, results):
if isinstance(text, Exception):
print(f"FAILED {path.name}: {text}")
else:
print(f"{path.name}: {text[:80]}...")Run it
Drop some .mp3 files into an audio/ folder next to your script, then run the program with the AI Fabric CLI:
blade run whisper_batch.pyAI Fabric launches one container per audio file (up to the fleet's capacity), and each transcription runs in parallel on its own GPU. You'll see a line per file land as the results come back in order, with any failures clearly marked rather than aborting the batch.
Take it for a spin
Everything above in one file, ready to run.
import bladekit
app = bladekit.app("whisper-batch")
# fabric/torch already ships torch, so don't reinstall it. Add Whisper + ffmpeg,
# plus datasets/soundfile so the script can fetch sample clips itself.
image = (
bladekit.Image.from_catalog("fabric/torch")
.pip_install("openai-whisper", "ffmpeg-python", "datasets", "soundfile")
.apt_install("ffmpeg")
)
@app.function(image=image, timeout=300)
def sample_clips(n: int = 6) -> list[bytes]:
"""Return a handful of short, stable public clips as WAV bytes, so the
tutorial is self-contained (no audio folder to supply)."""
import io, soundfile as sf
from datasets import load_dataset, Audio
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
# decode=False avoids datasets' torchcodec backend; soundfile reads the FLAC.
ds = ds.cast_column("audio", Audio(decode=False))
clips = []
for i in range(min(n, len(ds))):
arr, sr = sf.read(io.BytesIO(ds[i]["audio"]["bytes"]))
buf = io.BytesIO()
sf.write(buf, arr, sr, format="WAV")
clips.append(buf.getvalue())
return clips
@app.function(gpu="RTX-A4500", image=image, timeout=600)
def transcribe(audio_bytes: bytes) -> str:
import tempfile, whisper
model = whisper.load_model("large-v3")
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
f.write(audio_bytes)
tmp_path = f.name
result = model.transcribe(tmp_path)
return result["text"]
@app.local_entrypoint()
def main():
# Each clip is sent to a separate GPU container in parallel.
clips = sample_clips.remote()
results = list(transcribe.map(clips, return_exceptions=True))
for i, text in enumerate(results):
if isinstance(text, Exception):
print(f"FAILED clip {i}: {text}")
else:
print(f"clip {i}: {text[:80]}...")Concepts used: Cold start, Run, GPU