Developer docs / AI Fabric tutorials

Container lifecycle hooks: @build, @enter, @exit

  • GPU
  • Class

Take it for a spin

The whole program in one file, ready to run.

lifecycle_hooks.py
import bladekit

app = bladekit.app("lifecycle-hooks")

# fabric/torch catalog image: CUDA + PyTorch + transformers already installed.
@app.cls(gpu="RTX-A4500", image="fabric/torch", timeout=600)
class Embedder:
    MODEL = "BAAI/bge-small-en-v1.5"

    @bladekit.build()
    def fetch_weights(self):
        # Runs ONCE at image-build time: download the weights into the image
        # layer so every container starts with them already on disk. Paying the
        # download cost once at build time instead of on every cold start.
        from huggingface_hub import snapshot_download
        snapshot_download(self.MODEL)

    @bladekit.enter()
    def load(self):
        # Runs ONCE per container, before the first call: move the model onto the
        # GPU. The weights are already local thanks to @build, so this is fast.
        from sentence_transformers import SentenceTransformer
        self.model = SentenceTransformer(self.MODEL, device="cuda")
        self.served = 0

    @app.method()
    def embed(self, texts: list[str]) -> list[list[float]]:
        self.served += len(texts)
        return self.model.encode(texts).tolist()

    @bladekit.exit()
    def flush(self):
        # Runs ONCE per container at graceful shutdown, after the last call
        # drains and before the process exits. Flush a cache, commit a Volume,
        # close a network handle, or emit a final metric. Best-effort: an
        # exception here is logged and never wedges shutdown.
        print(f"container served {self.served} texts before exit")


@app.local_entrypoint()
def main():
    vecs = Embedder().embed.remote(["hello world", "AI Fabric lifecycle hooks"])
    print(len(vecs), "embeddings")

Run it

  1. Save as lifecycle_hooks.py and run blade run lifecycle_hooks.py.
  2. @bladekit.build runs once at image-build time to bake the model weights into the image layer. Every container then starts with the weights already present, so there is no per-cold-start download.
  3. @bladekit.enter runs once per container before the first call: it moves the (already-local) model onto the GPU. It re-runs on a fresh instance after a blade serve hot reload.
  4. Each call to embed reuses the same loaded model on the same container.
  5. @bladekit.exit runs once per container at graceful teardown, after the last call drains and before the process exits. It is the place to flush caches, commit a Volume, or close handles. All exit hooks run in definition order; a failing one is logged and the rest still run.

Concepts used: Cold start, Replica, Scale-to-zero, Endpoint

Contact the team

Loading the form…