Developer docs / AI Fabric tutorials
Container lifecycle hooks: @build, @enter, @exit
- GPU
- Class
Take it for a spin
The whole program in one file, ready to run.
import bladekit
app = bladekit.app("lifecycle-hooks")
# fabric/torch catalog image: CUDA + PyTorch + transformers already installed.
@app.cls(gpu="RTX-A4500", image="fabric/torch", timeout=600)
class Embedder:
MODEL = "BAAI/bge-small-en-v1.5"
@bladekit.build()
def fetch_weights(self):
# Runs ONCE at image-build time: download the weights into the image
# layer so every container starts with them already on disk. Paying the
# download cost once at build time instead of on every cold start.
from huggingface_hub import snapshot_download
snapshot_download(self.MODEL)
@bladekit.enter()
def load(self):
# Runs ONCE per container, before the first call: move the model onto the
# GPU. The weights are already local thanks to @build, so this is fast.
from sentence_transformers import SentenceTransformer
self.model = SentenceTransformer(self.MODEL, device="cuda")
self.served = 0
@app.method()
def embed(self, texts: list[str]) -> list[list[float]]:
self.served += len(texts)
return self.model.encode(texts).tolist()
@bladekit.exit()
def flush(self):
# Runs ONCE per container at graceful shutdown, after the last call
# drains and before the process exits. Flush a cache, commit a Volume,
# close a network handle, or emit a final metric. Best-effort: an
# exception here is logged and never wedges shutdown.
print(f"container served {self.served} texts before exit")
@app.local_entrypoint()
def main():
vecs = Embedder().embed.remote(["hello world", "AI Fabric lifecycle hooks"])
print(len(vecs), "embeddings")Run it
- Save as
lifecycle_hooks.pyand runblade run lifecycle_hooks.py. @bladekit.buildruns once at image-build time to bake the model weights into the image layer. Every container then starts with the weights already present, so there is no per-cold-start download.@bladekit.enterruns once per container before the first call: it moves the (already-local) model onto the GPU. It re-runs on a fresh instance after ablade servehot reload.- Each call to
embedreuses the same loaded model on the same container. @bladekit.exitruns once per container at graceful teardown, after the last call drains and before the process exits. It is the place to flush caches, commit a Volume, or close handles. All exit hooks run in definition order; a failing one is logged and the rest still run.
Concepts used: Cold start, Replica, Scale-to-zero, Endpoint