Developer docs / AI Fabric tutorials
Pet art generator (Flux + Gradio)
- GPU
- Volume
- Web endpoint
- Image DSL
Take it for a spin
The whole program in one file, ready to run.
import bladekit
import io
app = bladekit.app("pet-art-flux")
hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)
image = (
# torch + CUDA are baked into the catalog image, so do not re-install torch.
bladekit.Image.from_catalog("fabric/torch")
.pip_install(
# sentencepiece is required by FluxPipeline's T5 tokenizer.
"diffusers>=0.30.0", "transformers",
"accelerate", "peft", "Pillow", "safetensors", "sentencepiece"
)
.env({"HF_HOME": "/hf-cache"})
)
@app.cls(
gpu="RTX-A4500",
image=image,
timeout=3600,
volumes={"/hf-cache": hf_cache},
# FLUX.1-schnell is a gated HF repo, so inject a token via a Secret.
secrets=[bladekit.Secret.from_name("hf-token")],
)
class PetArtApp:
def __init__(self):
import torch
from diffusers import FluxPipeline
self.pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-schnell",
torch_dtype=torch.bfloat16,
)
# The 12B transformer alone is ~24 GB in bf16, over the A4500's ~20 GB,
# so enable_model_cpu_offload still OOMs at generation. Sequential offload
# streams each submodule through the GPU (a few GB of VRAM, slower). Do
# NOT also call .to("cuda"); the two are mutually exclusive.
self.pipe.enable_sequential_cpu_offload()
@app.web_endpoint(methods=["GET", "POST"], min_replicas=1)
def ui(self, item: dict) -> bytes:
prompt = item.get("prompt", "a golden retriever as a renaissance painting")
result = self.pipe(prompt, num_inference_steps=4)
buf = io.BytesIO()
result.images[0].save(buf, format="PNG")
return buf.getvalue()Run it
- Deploy:
blade deploy pet_art_flux.py. - The endpoint is served at
<app-url>/ui(the method name), not the bare URL. POST a prompt:curl -X POST <app-url>/ui -H 'Authorization: Bearer <FABRIC_TOKEN>' -d '{"prompt": "my golden retriever as a knight"}' -o art.png. Endpoints are private by default, so pass your AI Fabric token (or deploy withpublic=Truefor an unauthenticated endpoint). - The FIRST request on a fresh
hf-cachetakes ~10-15 min (the ~24 GB download runs inside__init__, then ~4 min of sequential-offload generation). Use a large curl--max-time(e.g. 900) and retry on an initial 503 while the replica wakes. Warm requests take ~4 min per image; that's expected, not a hang. - There is no local entrypoint: an
@app.clsis a remote service, so instantiating it in alocal_entrypointwould import torch and load the model on your laptop. Drive it through the deployed endpoint instead. - To fine-tune on specific pet photos, add a LoRA training step (see the Flux LoRA fine-tuning tutorial) and load the adapter in
__init__. min_replicas=1keeps the replica warm so the UI responds instantly.
Concepts used: Cold start, Endpoint, Replica, GPU