Model Deployment with FastAPI
Deploy ML models as production REST APIs using FastAPI, Docker, and best practices for monitoring and scaling.
Deployment Architecture
1. FastAPI Model Server
2. Request Validation
3. Docker Containerization
# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY model.pkl .
COPY app.py .
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
```txt
# requirements.txt
fastapi==0.109.0
uvicorn[standard]==0.27.0
joblib==1.3.2
numpy==1.26.3
scikit-learn==1.4.0
pydantic==2.5.3
prometheus-fastapi-instrumentator==6.1.0
```yaml
# docker-compose.yml
services:
api:
build: .
ports:
- "8000:8000"
environment:
- MODEL_PATH=/app/model.pkl
volumes:
- ./models:/app/models
deploy:
replicas: 3
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
prometheus:
image: prom/prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
image: grafana/grafana
ports:
- "3000:3000"
4. Model Serialization Formats
5. Async and Batch Inference
6. Production Checklist
- Input validation (Pydantic models)
- Error handling (try/except, proper HTTP codes)
- Health check endpoint (
/health) - Logging (structured JSON logs)
- Monitoring (latency, throughput, error rates)
- Rate limiting (prevent abuse)
- Authentication (API keys, JWT)
- CORS configuration
- Model versioning in responses
- Graceful shutdown handling
Key Takeaways
- FastAPI provides async support, automatic docs, and type safety
- Docker ensures reproducible deployments across environments
- Health checks enable orchestrators to manage container lifecycle
- Monitoring is essential — track latency, errors, and data drift