Python Cloud Deployment — AWS, GCP & Serverless
Cloud deployment makes your Python apps accessible from anywhere with automatic scaling. This tutorial covers serverless deployment, container platforms, and CI/CD pipelines.
Learning Objectives
-
Deploy to AWS Lambda with Mangum
-
Use Google Cloud Run and Azure Functions
-
Configure CI/CD pipelines with GitHub Actions
-
Manage cloud resources with boto3
-
Understand serverless vs container trade-offs
What is Cloud Deployment?
Local Development Cloud Deployment
+--------------+ +--------------------------+
| Your Laptop | | Cloud (AWS/GCP/Azure) |
| | | |
| main.py | ---? | +------------------+ |
| app.db | | | Lambda/Cloud Run | |
| config.env | | | (auto-scaling) | |
| | | +------------------+ |
| localhost | | Users worldwide access |
+--------------+ +--------------------------+
Serverless means you don't manage servers. You upload your code, and the cloud provider runs it, scales it, and charges you only for actual usage.
Serverless vs Container Deployment
| Feature | Serverless (Lambda/Cloud Run) | Containers (ECS/EKS) |
|---------|------------------------------|----------------------|
| Scaling | Automatic, instant | Manual or auto-scaling |
| Cold starts | Yes (first request delay) | No |
| Cost model | Pay per request/invocation | Pay for running instances |
| Max execution time | 15 min (Lambda) | Unlimited |
| Memory limit | 10GB (Lambda) | Unlimited |
| Best for | APIs, event handlers, jobs | Long-running services, ML |
| Complexity | Low | Medium to High |
| Startup cost | Free tier generous | Always running cost |
| State | Stateless | Can be stateful |
| Custom runtime | Limited | Full control |
When to Choose What
Use Serverless When: Use Containers When:
+- Event-driven +- Long-running processes
+- Intermittent traffic +- Consistent high traffic
+- Simple APIs +- Complex architectures
+- Quick prototypes +- ML model serving
+- Scheduled tasks +- Need full OS control
AWS Lambda with FastAPI
AWS Lambda runs Python functions in response to events (HTTP requests, file uploads, scheduled tasks). Mangum bridges FastAPI and Lambda.
Step 1: Install Dependencies
pip install fastapi mangum uvicorn
Step 2: Write Your API
from fastapi import FastAPI
from pydantic import BaseModel
app = FastAPI()
class Item(BaseModel):
name: str
price: float
description: str = ""
@app.get("/")
def root():
return {"message": "Hello from AWS Lambda"}
@app.get("/health")
def health():
return {"status": "healthy"}
@app.post("/items")
def create_item(item: Item):
return {"status": "created", "item": item}
@app.get("/items/{item_id}")
def get_item(item_id: int):
return {"item_id": item_id, "name": "Sample Item"}
Step 3: Create Lambda Handler
# handler.py
from mangum import Mangum
from main import app
# Mangum adapts FastAPI for Lambda
handler = Mangum(app)
# Lambda sends events like this:
# {
# "httpMethod": "GET",
# "path": "/items/42",
# "headers": {...},
# "body": null
# }
#
# Mangum converts this to a FastAPI request,
# runs your route, and converts the response back.
Step 4: Deploy with AWS CLI
# Install AWS CLI
pip install awscli
aws configure # Set up credentials
# Create deployment package
zip -r function.zip . -x "*.git*" "__pycache__/*" "*.pyc"
# Create Lambda function
aws lambda create-function \
--function-name my-fastapi \
--runtime python3.11 \
--handler handler.handler \
--zip-file fileb://function.zip \
--role arn:aws:iam::YOUR_ACCOUNT:role/lambda-role \
--timeout 30 \
--memory-size 256
# Create API Gateway trigger
aws apigateway create-rest-api --name "my-api"
# (Configure routes to point to Lambda)
Lambda with API Gateway (SAM Template)
# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
FastApiFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: .
Handler: handler.handler
Runtime: python3.11
Timeout: 30
MemorySize: 256
Events:
Api:
Type: Api
Properties:
Path: /{proxy+}
Method: ANY
# Deploy with SAM
sam build
sam deploy --guided
Google Cloud Run
Cloud Run runs containers (not just functions) with automatic scaling to zero. Perfect for containerized FastAPI apps.
Dockerfile for Cloud Run
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Cloud Run requires listening on PORT env var
ENV PORT=8080
EXPOSE $PORT
CMD exec uvicorn main:app --host 0.0.0.0 --port $PORT
Deploy to Cloud Run
# Build and deploy (uses Google Cloud Build)
gcloud run deploy my-api \
--source . \
--platform managed \
--region us-central1 \
--allow-unauthenticated \
--memory 512Mi \
--cpu 1
# Set environment variables
gcloud run services update my-api \
--set-env-vars DATABASE_URL=...,API_KEY=...
Cloud Run with Cloud SQL
# service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: my-api
spec:
template:
spec:
containers:
- image: gcr.io/my-project/my-api
env:
- name: DATABASE_URL
value: "postgresql://..."
volumeMounts:
- name: cloudsql
mountPath: /cloudsql
metadata:
annotations:
run.googleapis.com/cloud-sql-instances: "project:region:instance"
Azure Functions
Azure Functions provides serverless compute with Python support.
function_app.py
import azure.functions as func
import json
app = func.FunctionApp()
@app.route(route="hello", auth_level=func.AuthLevel.ANONYMOUS)
def hello(req: func.HttpRequest) -> func.HttpResponse:
name = req.params.get('name', 'World')
return func.HttpResponse(
json.dumps({"message": f"Hello, {name}!"}),
mimetype="application/json"
)
@app.timer_trigger(schedule="0 0 */6 * * *", arg_name="myTimer")
def scheduled_task(myTimer: func.TimerRequest) -> None:
# Runs every 6 hours
print("Scheduled task executed")
# Create function project
func init my-function-app --python
# Add function
func new --name hello --template "HTTP trigger"
# Deploy
func azure functionapp publish my-function-app
CI/CD with GitHub Actions
Complete Pipeline: Test, Build, Deploy
# .github/workflows/deploy.yml
name: Deploy to AWS
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run linting
run: ruff check .
- name: Run type check
run: mypy src/
- name: Run tests
run: pytest tests/ -v --cov=src --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v3
build:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Build Docker image
run: docker build -t myapp:${{ github.sha }} .
- name: Push to ECR
uses: aws-actions/amazon-ecr-login@v2
with:
registry: ${{ vars.AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com
- name: Push image
run: |
docker tag myapp:${{ github.sha }} $ECR_REGISTRY/myapp:${{ github.sha }}
docker push $ECR_REGISTRY/myapp:${{ github.sha }}
deploy:
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- name: Deploy to Lambda
uses: appleboy/lambda-action@v1.0.0
with:
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
region: us-east-1
function_name: my-fastapi
zip_file: function.zip
How CI/CD Works
Push to GitHub
|
v
+-------------+
| Run Tests | ?-- pytest, mypy, linting
+------+------+
| Tests pass
v
+-------------+
| Build Image | ?-- Docker build + push to registry
+------+------+
|
v
+-------------+
| Deploy | ?-- Update Lambda/Cloud Run
+------+------+
|
v
+-------------+
| Live! | ?-- Users can access the new version
+-------------+
Environment Variables and Secrets
AWS Secrets Manager
import boto3
import json
def get_secret(secret_name):
client = boto3.client('secretsmanager')
response = client.get_secret_value(SecretId=secret_name)
return json.loads(response['SecretString'])
# Usage
db_secret = get_secret('prod/myapp/database')
DATABASE_URL = f"postgresql://{db_secret['user']}:{db_secret['password']}@{db_secret['host']}:5432/mydb"
Environment Variables in Lambda
# Set via CLI
aws lambda update-function-configuration \
--function-name my-fastapi \
--environment "Variables={DATABASE_URL=...,DEBUG=false}"
# Set via console
# Lambda -> Configuration -> Environment variables
Google Cloud Secret Manager
from google.cloud import secretmanager
def access_secret(project_id, secret_id, version_id="latest"):
client = secretmanager.SecretManagerServiceClient()
name = f"projects/{project_id}/secrets/{secret_id}/versions/{version_id}"
response = client.access_secret_version(request={"name": name})
return response.payload.data.decode("UTF-8")
Managing Cloud Resources with boto3
import boto3
# S3 — File Storage
s3 = boto3.client('s3')
s3.upload_file('local_file.txt', 'my-bucket', 'remote_file.txt')
s3.download_file('my-bucket', 'remote_file.txt', 'downloaded.txt')
# List bucket contents
response = s3.list_objects_v2(Bucket='my-bucket')
for obj in response.get('Contents', []):
print(obj['Key'])
# DynamoDB — NoSQL Database
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Users')
table.put_item(Item={'id': '1', 'name': 'Alice'})
response = table.get_item(Key={'id': '1'})
# SQS — Message Queue
sqs = boto3.client('sqs')
sqs.send_message(QueueUrl='https://...', MessageBody='Hello')
# CloudWatch — Monitoring
cloudwatch = boto3.client('cloudwatch')
cloudwatch.put_metric_data(
Namespace='MyApp',
MetricData=[{
'MetricName': 'RequestCount',
'Value': 1,
'Unit': 'Count'
}]
)
Common Mistakes
| Mistake | Problem | Solution |
|---------|---------|----------|
| Hardcoding secrets | Exposed credentials | Use Secrets Manager |
| Not setting timeouts | Functions hang forever | Set appropriate timeout |
| Ignoring cold starts | Slow first responses | Use provisioned concurrency |
| No logging | Can't debug issues | Use CloudWatch/Cloud Logging |
| No health checks | Can't detect failures | Add health endpoint |
| Over-provisioning memory | Unnecessary cost | Right-size based on usage |
| No CI/CD | Manual deployment errors | Automate with GitHub Actions |
Key Takeaways
-
Lambda is great for event-driven, short-running tasks
-
Use Mangum to run FastAPI on Lambda
-
Cloud Run offers container flexibility with serverless scaling
-
Package dependencies with your code in zip file
-
Use GitHub Actions for automated testing and deployment
-
Store secrets in AWS Secrets Manager, not in code
-
Monitor with CloudWatch or Cloud Logging
-
Start with serverless — scale to containers if needed
-
Use SAM or Serverless Framework for infrastructure as code
-
Always set appropriate timeouts and memory limits