Start with the workload
AI Fabric is for your model or Python code. Models is for calling models that are already hosted behind an API. Choose the former when you need control over model weights, dependencies or the work around inference.
AI Fabric is opening gradually. This guide helps you prepare for onboarding. To see what running code looks like, follow the AI Fabric getting started. The GPU options are RTX A4500 (20 GB) and RTX 2000 Ada (16 GB). Confirm the configuration for your workload during onboarding.
Package the model and its dependencies
Keep the model identifier or weights location, Python dependencies and inference entry point together. Record the model licence and any access requirements for its weights. Pin the dependencies you have actually tested.
A useful project structure is:
my-model/
requirements.txt
predict.py
sample-request.json
README.md
This is a suggested project layout, not an AI Fabric SDK contract. Your predict.py should load the model and accept the input shape your application will send.
Size the workload before choosing a GPU
| Prepare | Why it matters |
|---|---|
| Model size, precision and expected input size | Establish the GPU memory requirement. |
| A representative sample request | Measure latency and output quality on the real workload. |
| Expected request rate or batch volume | Decide how much concurrent work the service must handle. |
| Model loading time | Understand how startup affects the first request after idle. |
| Runtime dependencies and weights access | Make deployment reproducible. |
| Required timeout and retry behaviour | Define how the application handles long or interrupted work. |
Check the deployment during onboarding
Confirm the supported Python runtime, GPU class, dependency installation and secrets configuration with the team. Deploy a small test workload first, then exercise its input and output contract before connecting it to production traffic.
For an interactive endpoint, measure cold and warm latency separately. For a batch pipeline, test a representative group of inputs and verify that retrying a job does not duplicate downstream work.
Confirm scaling and costs
Discuss the concurrency limit, startup behaviour, idle policy and billing unit for your workload. The AI Fabric product is designed around autoscaling and scale to zero; the available settings and commercial terms are confirmed when access opens.
Models token prices and their dynamic pricing are not an AI Fabric GPU quote. Dedicated model endpoints described in the inference API documentation are also a separate billing surface.
Ready to bring your model?
Join the AI Fabric waitlist with your model, memory requirement, representative input and expected traffic. If you need a hosted model API now, start with Models.