Developer docs / AI Fabric tutorials

TTS with Chatterbox

  • GPU
  • Volume
  • Class
  • Image DSL

Take it for a spin

The whole program in one file, ready to run.

tts_chatterbox.py
import bladekit
import io

app = bladekit.app("tts-chatterbox")

# Model weights are cached here after the first download (~2 GB).
hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

# chatterbox-tts pins numpy<2 and torch==2.6.0 (no Python 3.13 wheels), so use a
# Python 3.12 base and let pip resolve chatterbox's own pins (do not add torch
# yourself, it would skew torchaudio out of sync).
image = (
    bladekit.Image.debian_slim(python_version="3.12")
    .apt_install("ffmpeg")
    # setuptools<81: chatterbox's perth watermarker imports pkg_resources,
    # which newer setuptools removed. Without it the watermarker import fails
    # silently and every request crashes with "'NoneType' object is not callable".
    .pip_install("chatterbox-tts", "soundfile", "hf_transfer", "setuptools<81")
    .env({"HF_HOME": "/hf-cache", "HF_HUB_ENABLE_HF_TRANSFER": "1"})
)

@app.cls(gpu="RTX-A4500", image=image, timeout=600, volumes={"/hf-cache": hf_cache})
class ChatterboxTTS:
    def __init__(self):
        from chatterbox.tts import ChatterboxTTS as CB

        # Weights come from the ResembleAI/chatterbox HF repo, cached in the Volume.
        self.model = CB.from_pretrained(device="cuda")

    # min_replicas=1 keeps a warm replica: model load takes longer than the
    # gateway wake window, so a scale-from-zero first request would time out.
    @app.web_endpoint(methods=["POST"], min_replicas=1)
    def synthesize(self, item: dict) -> bytes:
        import soundfile as sf

        text = item.get("text", "Hello from Chatterbox!")
        wav = self.model.generate(text)  # (1, N) float tensor
        buf = io.BytesIO()
        # Chatterbox outputs 24 kHz mono audio (model.sr == 24000).
        sf.write(buf, wav.squeeze().cpu().numpy(), samplerate=self.model.sr, format="WAV")
        return buf.getvalue()

Run it

  1. chatterbox-tts pins torch 2.6 and numpy<2 (no Python 3.13 wheels). The image pins Python 3.12, and AI Fabric requires the client Python to MATCH: uv venv -p 3.12 .venv312 && uv pip install --python .venv312/bin/python bladekit, then .venv312/bin/blade deploy tts_chatterbox.py.
  2. setuptools<81 is load-bearing: chatterbox's perth watermarker imports pkg_resources (removed in newer setuptools) and the failure only surfaces at request time as "'NoneType' object is not callable".
  3. Let pip resolve chatterbox's own pins. Do not add torch yourself: it would skew torchaudio out of sync.
  4. First request after deploy takes ~2-3 min (image pull + ~2 GB of weights into the Volume); min_replicas=1 keeps the replica warm afterwards.
  5. POST text, get 24 kHz WAV bytes: curl -X POST <app-url>/synthesize -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"text": "..."}' -o speech.wav.

Concepts used: Cold start, Replica, Endpoint, GPU

Contact the team

Loading the form…