Developer docs / AI Fabric tutorials
Text-to-speech web endpoint
- GPU
- Web endpoint
- Image DSL
Take it for a spin
The whole program in one file, ready to run.
import bladekit
app = bladekit.app("tts-endpoint")
# "coqui-tts" is the maintained fork of the original (unmaintained) "TTS"
# package. Unlike "TTS" it supports modern Python, so no 3.11 base is needed.
# fabric/torch already ships CUDA torch, so don't reinstall it. torchaudio must
# come from the SAME cu128 index as the image's torch: the default PyPI wheel
# links against a different CUDA build and fails to load.
image = (
bladekit.Image.from_catalog("fabric/torch")
.apt_install("ffmpeg", "espeak-ng")
.pip_install(
"torchaudio==2.11.0",
index_url="https://download.pytorch.org/whl/cu128",
)
# [codec]: torch >= 2.9 requires torchcodec for audio IO.
# transformers<5: coqui-tts uses isin_mps_friendly, removed in v5.
.pip_install("coqui-tts[codec]", "transformers<5")
)
_tts = None
@app.web_endpoint(methods=["POST"], max_replicas=2)
@app.function(gpu="RTX-A4500", image=image, timeout=300)
@app.concurrent(max_inputs=4)
def synthesize(item: dict) -> bytes:
global _tts
import os
import tempfile
from TTS.api import TTS as TTSModel
# Load the model once per container; later requests reuse it.
if _tts is None:
_tts = TTSModel("tts_models/en/ljspeech/vits").to("cuda")
text = item.get("text", "Hello from AI Fabric!")
# Returning bytes from a web endpoint sends them as an octet-stream body.
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "out.wav")
_tts.tts_to_file(text=text, file_path=path)
with open(path, "rb") as f:
return f.read()Run it
- Deploy:
blade deploy tts_endpoint.py. - The maintained
coqui-ttsfork supports modern Python, so no 3.11 base is needed (the originalTTSpackage is unmaintained). Two pins are load-bearing:transformers<5(coqui uses an API removed in transformers v5) and the[codec]extra (torch ≥ 2.9 requirestorchcodecfor audio IO). torchaudiomust come from the samecu128index as the image'storch. The default PyPI wheel links a different CUDA build and fails to load.- POST text, get WAV bytes:
curl -X POST <app-url>/synthesize -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"text": "Hello!"}' -o speech.wav. The first request on a fresh replica loads the VITS model; warm requests are fast. - Swap the model id for other voices/languages, e.g. XTTS-v2 for voice cloning. Add
COQUI_TOS_AGREED=1to the image env for XTTS.
Concepts used: Cold start, Endpoint, Replica, Scale-to-zero