Developer docs / AI Fabric tutorials
Mount an S3 bucket as a filesystem path
- Secret
- Image DSL
Take it for a spin
The whole program in one file, ready to run.
import bladekit
import pathlib
app = bladekit.app("s3-mount")
# The AI Fabric node mounts the bucket on the host and bind-mounts it into your
# container. You don't install or run s3fs yourself, and your code never needs
# any special privilege. Just bring libraries to parse what you read:
# pandas + pyarrow for parquet here.
image = (
bladekit.Image.debian_slim(python_version="3.12")
.pip_install("pandas", "pyarrow")
)
# Credentials come ONLY from a Secret, never inline. The secret holds
# AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (+ optional AWS_SESSION_TOKEN).
aws_secret = bladekit.Secret.from_name("aws-credentials")
@app.function(
image=image,
timeout=300,
volumes={
# Mount the bucket read-only at /mnt/data, scoped to one key prefix.
"/mnt/data": bladekit.CloudBucketMount(
"my-data-bucket",
secret=aws_secret,
read_only=True,
key_prefix="raw/2024/",
),
},
)
def process_s3_data() -> dict:
import pandas as pd
# The bucket is a normal directory now: glob and read like local files.
root = pathlib.Path("/mnt/data")
parquet_files = sorted(root.glob("*.parquet"))[:5]
dfs = [pd.read_parquet(f) for f in parquet_files]
if not dfs:
return {"files_read": 0, "total_rows": 0, "columns": []}
combined = pd.concat(dfs)
return {
"files_read": len(dfs),
"total_rows": len(combined),
"columns": list(combined.columns),
}
@app.local_entrypoint()
def main():
result = process_s3_data.remote()
print(f"Read {result['files_read']} files, {result['total_rows']} rows")Run it
- Store your bucket credentials:
blade secret create aws-credentials AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... bladekit.CloudBucketMountmounts an external bucket at a path. It rides in the samevolumes=map as abladekit.Volume, keyed by mount path.- The wire spec carries only the secret NAME. The control plane resolves it and injects the keys as env vars; the runner reads those to FUSE-mount the bucket. A credential value never appears in the run body, argv, or logs.
read_only=Truemounts without write access;key_prefix="raw/2024/"mounts only that sub-path. For non-AWS stores (R2, MinIO, Scaleway) passbucket_endpoint_url=.- Run
blade run s3_mount.py. Your code sees/mnt/dataas a directory: no boto3, no download-to-disk.
Concepts used: Run, Secret