Developer docs / AI Fabric tutorials

Pet art generator (Flux + Gradio)

  • GPU
  • Volume
  • Web endpoint
  • Image DSL

Take it for a spin

The whole program in one file, ready to run.

pet_art_flux.py
import bladekit
import io

app = bladekit.app("pet-art-flux")

hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

image = (
    # torch + CUDA are baked into the catalog image, so do not re-install torch.
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install(
        # sentencepiece is required by FluxPipeline's T5 tokenizer.
        "diffusers>=0.30.0", "transformers",
        "accelerate", "peft", "Pillow", "safetensors", "sentencepiece"
    )
    .env({"HF_HOME": "/hf-cache"})
)

@app.cls(
    gpu="RTX-A4500",
    image=image,
    timeout=3600,
    volumes={"/hf-cache": hf_cache},
    # FLUX.1-schnell is a gated HF repo, so inject a token via a Secret.
    secrets=[bladekit.Secret.from_name("hf-token")],
)
class PetArtApp:
    def __init__(self):
        import torch
        from diffusers import FluxPipeline

        self.pipe = FluxPipeline.from_pretrained(
            "black-forest-labs/FLUX.1-schnell",
            torch_dtype=torch.bfloat16,
        )
        # The 12B transformer alone is ~24 GB in bf16, over the A4500's ~20 GB,
        # so enable_model_cpu_offload still OOMs at generation. Sequential offload
        # streams each submodule through the GPU (a few GB of VRAM, slower). Do
        # NOT also call .to("cuda"); the two are mutually exclusive.
        self.pipe.enable_sequential_cpu_offload()

    @app.web_endpoint(methods=["GET", "POST"], min_replicas=1)
    def ui(self, item: dict) -> bytes:
        prompt = item.get("prompt", "a golden retriever as a renaissance painting")
        result = self.pipe(prompt, num_inference_steps=4)
        buf = io.BytesIO()
        result.images[0].save(buf, format="PNG")
        return buf.getvalue()

Run it

  1. Deploy: blade deploy pet_art_flux.py.
  2. The endpoint is served at <app-url>/ui (the method name), not the bare URL. POST a prompt: curl -X POST <app-url>/ui -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"prompt": "my golden retriever as a knight"}' -o art.png. Endpoints are private by default, so pass your AI Fabric token (or deploy with public=True for an unauthenticated endpoint).
  3. The FIRST request on a fresh hf-cache takes ~10-15 min (the ~24 GB download runs inside __init__, then ~4 min of sequential-offload generation). Use a large curl --max-time (e.g. 900) and retry on an initial 503 while the replica wakes. Warm requests take ~4 min per image; that's expected, not a hang.
  4. There is no local entrypoint: an @app.cls is a remote service, so instantiating it in a local_entrypoint would import torch and load the model on your laptop. Drive it through the deployed endpoint instead.
  5. To fine-tune on specific pet photos, add a LoRA training step (see the Flux LoRA fine-tuning tutorial) and load the adapter in __init__.
  6. min_replicas=1 keeps the replica warm so the UI responds instantly.

Concepts used: Cold start, Endpoint, Replica, GPU

Contact the team

Loading the form…