Developer docs / AI Fabric tutorials

Animate an image (video generation)

  • GPU
  • Volume
  • Image DSL

A still image is one frame; a video is many frames that agree with each other over time. AnimateDiff does that: it bolts a temporal motion module onto an ordinary Stable Diffusion model, so the same prompt machinery that paints a picture now paints a short, coherent clip.

In this tutorial you'll turn a text prompt into a short MP4 on a GPU. The function returns raw video bytes, so you can write the clip straight to disk or stream it from a web endpoint, and the weights are cached on a Volume so the download only happens once.

Before you start

You need the AI Fabric CLI installed and authenticated: run blade setup once. The function requests a single RTX A4500 and builds on fabric/torch, the catalog image with CUDA and PyTorch already baked in, so there's nothing to build locally.

Create the app and a weights Volume

Create a file called image_animate.py. Alongside the App, declare a Volume, storage that survives across runs. AnimateDiff pulls down both a base SD model and a motion adapter; caching them here means you pay that download exactly once.

image_animate.py
import bladekit
import io

app = bladekit.app("image-animate")

hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

Define the image

Start from fabric/torch and add the diffusion stack. Two extras matter here: diffusers>=0.28.0 is the version that ships AnimateDiffPipeline, and imageio[ffmpeg] gives you the encoder that turns frames into an MP4. Pointing HF_HOME at /hf-cache is what routes the downloaded weights into the Volume.

Python
image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install(
        "torch", "diffusers>=0.28.0", "transformers",
        "accelerate", "safetensors", "imageio[ffmpeg]", "Pillow"
    )
    .env({"HF_HOME": "/hf-cache"})
)

Load the motion adapter and pipeline

Request the GPU and mount the Volume, then build the pipeline behind a _pipe global so it stays resident across calls. AnimateDiff is assembled from two pieces: a MotionAdapter (the temporal module) and a base image model it animates (here epiCRealism). Swapping the scheduler to DDIMScheduler gives the smoother sampling AnimateDiff expects.

Python
_pipe = None

@app.function(gpu="RTX-A4500", image=image, timeout=1800, volumes={"/hf-cache": hf_cache})
def animate(prompt: str, num_frames: int = 16) -> bytes:
    global _pipe
    import torch, imageio, numpy as np
    from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter

    if _pipe is None:
        adapter = MotionAdapter.from_pretrained(
            "guoyww/animatediff-motion-adapter-v1-5-2",
            torch_dtype=torch.float16,
        )
        _pipe = AnimateDiffPipeline.from_pretrained(
            "emilianJR/epiCRealism",
            motion_adapter=adapter,
            torch_dtype=torch.float16,
        ).to("cuda")
        _pipe.scheduler = DDIMScheduler.from_config(_pipe.scheduler.config)

The first call (when _pipe is None) downloads both models and moves them onto the GPU; every later call in the warm container reuses them. That one-time load is why the first clip is slow and the rest are fast.

Render the frames to MP4

Generating returns a list of frames. Feed them through imageio.mimsave to encode an MP4 at 8 fps in memory, then return the raw bytes. Anyone calling animate gets back a valid video file they can save or serve directly.

Python
    frames = _pipe(prompt, num_frames=num_frames).frames[0]
    buf = io.BytesIO()
    imageio.mimsave(buf, [np.array(f) for f in frames], format="mp4", fps=8)
    return buf.getvalue()

Call it from your machine

The local_entrypoint runs on your laptop. animate.remote(...) ships the prompt to the GPU, runs generation there, and hands the MP4 bytes back, which you write to a local file.

Python
@app.local_entrypoint()
def main():
    import pathlib
    mp4 = animate.remote("a cat walking through autumn leaves, cinematic")
    pathlib.Path("animation.mp4").write_bytes(mp4)
    print("Saved animation.mp4")

Run it

Run the program with the AI Fabric CLI:

Terminal
blade run image_animate.py

The first run downloads the base model and motion adapter into the Volume (the cold start), then generates the clip. When it finishes you'll have a playable animation.mp4 on disk. The returned bytes are a valid MP4, so the same function drops straight behind a web endpoint when you want it as a URL.

Take it for a spin

Everything above in one file, ready to run.

image_animate.py
import bladekit
import io

app = bladekit.app("image-animate")

hf_cache = bladekit.Volume.from_name("hf-cache", create_if_missing=True)

image = (
    bladekit.Image.from_catalog("fabric/torch")
    .pip_install(
        "torch", "diffusers>=0.28.0", "transformers",
        "accelerate", "safetensors", "imageio[ffmpeg]", "Pillow"
    )
    .env({"HF_HOME": "/hf-cache"})
)

_pipe = None

@app.function(gpu="RTX-A4500", image=image, timeout=1800, volumes={"/hf-cache": hf_cache})
def animate(prompt: str, num_frames: int = 16) -> bytes:
    global _pipe
    import torch, imageio, numpy as np
    from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter

    if _pipe is None:
        adapter = MotionAdapter.from_pretrained(
            "guoyww/animatediff-motion-adapter-v1-5-2",
            torch_dtype=torch.float16,
        )
        _pipe = AnimateDiffPipeline.from_pretrained(
            "emilianJR/epiCRealism",
            motion_adapter=adapter,
            torch_dtype=torch.float16,
        ).to("cuda")
        _pipe.scheduler = DDIMScheduler.from_config(_pipe.scheduler.config)

    frames = _pipe(prompt, num_frames=num_frames).frames[0]
    buf = io.BytesIO()
    imageio.mimsave(buf, [np.array(f) for f in frames], format="mp4", fps=8)
    return buf.getvalue()

@app.local_entrypoint()
def main():
    import pathlib
    mp4 = animate.remote("a cat walking through autumn leaves, cinematic")
    pathlib.Path("animation.mp4").write_bytes(mp4)
    print("Saved animation.mp4")

Concepts used: Cold start, GPU, Image build

Contact the team

Loading the form…