Developer docs / AI Fabric tutorials

Mount an S3 bucket as a filesystem path

  • Secret
  • Image DSL

Take it for a spin

The whole program in one file, ready to run.

s3_mount.py
import bladekit
import pathlib

app = bladekit.app("s3-mount")

# The AI Fabric node mounts the bucket on the host and bind-mounts it into your
# container. You don't install or run s3fs yourself, and your code never needs
# any special privilege. Just bring libraries to parse what you read:
# pandas + pyarrow for parquet here.
image = (
    bladekit.Image.debian_slim(python_version="3.12")
    .pip_install("pandas", "pyarrow")
)

# Credentials come ONLY from a Secret, never inline. The secret holds
# AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (+ optional AWS_SESSION_TOKEN).
aws_secret = bladekit.Secret.from_name("aws-credentials")

@app.function(
    image=image,
    timeout=300,
    volumes={
        # Mount the bucket read-only at /mnt/data, scoped to one key prefix.
        "/mnt/data": bladekit.CloudBucketMount(
            "my-data-bucket",
            secret=aws_secret,
            read_only=True,
            key_prefix="raw/2024/",
        ),
    },
)
def process_s3_data() -> dict:
    import pandas as pd

    # The bucket is a normal directory now: glob and read like local files.
    root = pathlib.Path("/mnt/data")
    parquet_files = sorted(root.glob("*.parquet"))[:5]

    dfs = [pd.read_parquet(f) for f in parquet_files]
    if not dfs:
        return {"files_read": 0, "total_rows": 0, "columns": []}

    combined = pd.concat(dfs)
    return {
        "files_read": len(dfs),
        "total_rows": len(combined),
        "columns": list(combined.columns),
    }

@app.local_entrypoint()
def main():
    result = process_s3_data.remote()
    print(f"Read {result['files_read']} files, {result['total_rows']} rows")

Run it

  1. Store your bucket credentials: blade secret create aws-credentials AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...
  2. bladekit.CloudBucketMount mounts an external bucket at a path. It rides in the same volumes= map as a bladekit.Volume, keyed by mount path.
  3. The wire spec carries only the secret NAME. The control plane resolves it and injects the keys as env vars; the runner reads those to FUSE-mount the bucket. A credential value never appears in the run body, argv, or logs.
  4. read_only=True mounts without write access; key_prefix="raw/2024/" mounts only that sub-path. For non-AWS stores (R2, MinIO, Scaleway) pass bucket_endpoint_url=.
  5. Run blade run s3_mount.py. Your code sees /mnt/data as a directory: no boto3, no download-to-disk.

Concepts used: Run, Secret

Contact the team

Loading the form…