Developer docs / AI Fabric tutorials

Text-to-speech web endpoint

  • GPU
  • Web endpoint
  • Image DSL

Take it for a spin

The whole program in one file, ready to run.

tts_endpoint.py
import bladekit

app = bladekit.app("tts-endpoint")

# "coqui-tts" is the maintained fork of the original (unmaintained) "TTS"
# package. Unlike "TTS" it supports modern Python, so no 3.11 base is needed.
# fabric/torch already ships CUDA torch, so don't reinstall it. torchaudio must
# come from the SAME cu128 index as the image's torch: the default PyPI wheel
# links against a different CUDA build and fails to load.
image = (
    bladekit.Image.from_catalog("fabric/torch")
    .apt_install("ffmpeg", "espeak-ng")
    .pip_install(
        "torchaudio==2.11.0",
        index_url="https://download.pytorch.org/whl/cu128",
    )
    # [codec]: torch >= 2.9 requires torchcodec for audio IO.
    # transformers<5: coqui-tts uses isin_mps_friendly, removed in v5.
    .pip_install("coqui-tts[codec]", "transformers<5")
)

_tts = None

@app.web_endpoint(methods=["POST"], max_replicas=2)
@app.function(gpu="RTX-A4500", image=image, timeout=300)
@app.concurrent(max_inputs=4)
def synthesize(item: dict) -> bytes:
    global _tts
    import os
    import tempfile
    from TTS.api import TTS as TTSModel

    # Load the model once per container; later requests reuse it.
    if _tts is None:
        _tts = TTSModel("tts_models/en/ljspeech/vits").to("cuda")

    text = item.get("text", "Hello from AI Fabric!")

    # Returning bytes from a web endpoint sends them as an octet-stream body.
    with tempfile.TemporaryDirectory() as tmp:
        path = os.path.join(tmp, "out.wav")
        _tts.tts_to_file(text=text, file_path=path)
        with open(path, "rb") as f:
            return f.read()

Run it

  1. Deploy: blade deploy tts_endpoint.py.
  2. The maintained coqui-tts fork supports modern Python, so no 3.11 base is needed (the original TTS package is unmaintained). Two pins are load-bearing: transformers<5 (coqui uses an API removed in transformers v5) and the [codec] extra (torch ≥ 2.9 requires torchcodec for audio IO).
  3. torchaudio must come from the same cu128 index as the image's torch. The default PyPI wheel links a different CUDA build and fails to load.
  4. POST text, get WAV bytes: curl -X POST <app-url>/synthesize -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"text": "Hello!"}' -o speech.wav. The first request on a fresh replica loads the VITS model; warm requests are fast.
  5. Swap the model id for other voices/languages, e.g. XTTS-v2 for voice cloning. Add COQUI_TOS_AGREED=1 to the image env for XTTS.

Concepts used: Cold start, Endpoint, Replica, Scale-to-zero

Contact the team

Loading the form…