Python Configuration — Settings & Environment

Configuration management keeps your app flexible across environments (development, staging, production). Hardcoding values creates security risks and deployment headaches. This tutorial covers environment variables, .env files, YAML/JSON/TOML config, Pydantic Settings, and the 12-factor app methodology.

Learning Objectives

• Use environment variables for configuration
• Load .env files with python-dotenv
• Create typed configuration classes with validation
• Parse YAML, JSON, and TOML configuration files
• Implement multi-environment configuration
• Apply 12-factor app principles

Why Configuration Management Matters

Development          Staging             Production
-----------          -------             ----------
DEBUG=True           DEBUG=False         DEBUG=False
DB=localhost         DB=staging-db       DB=production-db
PORT=8000            PORT=80             PORT=80
SECRET=dev-key       SECRET=staging-key  SECRET=prod-key
LOG_LEVEL=DEBUG      LOG_LEVEL=INFO      LOG_LEVEL=WARNING

Without proper configuration, you'd need to change code for each environment — dangerous and error-prone.

Environment Variables

import os

# Read from environment (set externally)
DATABASE_URL = os.environ.get("DATABASE_URL", "sqlite:///default.db")
DEBUG = os.environ.get("DEBUG", "false").lower() == "true"
PORT = int(os.environ.get("PORT", "8000"))
SECRET_KEY = os.environ.get("SECRET_KEY")

# Validate required variables
if not SECRET_KEY:
    raise ValueError("SECRET_KEY environment variable is required")

# Type conversion
MAX_CONNECTIONS = int(os.environ.get("MAX_CONNECTIONS", "10"))
TIMEOUT = float(os.environ.get("TIMEOUT", "30.0"))

# List from comma-separated string
ALLOWED_ORIGINS = os.environ.get("ALLOWED_ORIGINS", "").split(",")

Setting Environment Variables

# Linux/Mac
export DATABASE_URL="postgresql://user:pass@localhost/mydb"
export DEBUG=true

# Windows (PowerShell)
$env:DATABASE_URL="postgresql://user:pass@localhost/mydb"
$env:DEBUG="true"

# Windows (CMD)
set DATABASE_URL=postgresql://user:pass@localhost/mydb
set DEBUG=true

# Inline for single command
DATABASE_URL="postgresql://user:pass@localhost/mydb" python app.py

.env Files with python-dotenv

# .env file (DO NOT commit to git!)
DATABASE_URL=postgresql://user:pass@localhost/mydb
DEBUG=true
SECRET_KEY=your-secret-key-here
PORT=8000
REDIS_URL=redis://localhost:6379
ALLOWED_ORIGINS=http://localhost:3000,http://localhost:8080
MAX_CONNECTIONS=10
TIMEOUT=30.0

# Load .env into environment
from dotenv import load_dotenv
import os

# Load .env file (overrides existing env vars by default)
load_dotenv()

# Load specific file
load_dotenv(".env.local")

# Override existing variables (use with caution)
load_dotenv(override=True)

# Now accessible via os.environ
DATABASE_URL = os.environ["DATABASE_URL"]
DEBUG = os.environ.get("DEBUG", "false").lower() == "true"

.env File Best Practices

# .env.example (committed to git — shows required variables without secrets)
DATABASE_URL=postgresql://user:password@localhost/dbname
DEBUG=true
SECRET_KEY=generate-a-real-secret-key
PORT=8000

# .gitignore (never commit real .env)
.env
.env.local
.env.production

YAML Configuration Files

# config.yaml
database:
  url: postgresql://user:pass@localhost/mydb
  pool_size: 10
  timeout: 30

server:
  host: 0.0.0.0
  port: 8000
  debug: false

logging:
  level: INFO
  format: json

redis:
  url: redis://localhost:6379
  ttl: 300

import yaml

def load_yaml_config(path: str) -> dict:
    with open(path, 'r') as f:
        return yaml.safe_load(f)

config = load_yaml_config("config.yaml")
print(config["database"]["url"])
print(config["server"]["port"])

Installing PyYAML

pip install pyyaml

JSON Configuration Files

# config.json
{
  "database": {
    "url": "postgresql://user:pass@localhost/mydb",
    "pool_size": 10,
    "timeout": 30
  },
  "server": {
    "host": "0.0.0.0",
    "port": 8000,
    "debug": false
  }
}

import json
from pathlib import Path

def load_json_config(path: str) -> dict:
    with open(path, 'r') as f:
        return json.load(f)

config = load_json_config("config.json")

TOML Configuration Files

# pyproject.toml (modern Python standard)
[tool.myapp]
debug = true
log_level = "INFO"

[tool.myapp.database]
url = "postgresql://user:pass@localhost/mydb"
pool_size = 10

# Python 3.11+ has built-in tomllib
import tomllib

with open("pyproject.toml", "rb") as f:
    config = tomllib.load(f)

# Python < 3.11 needs tomli
# pip install tomli
import tomli

with open("config.toml", "rb") as f:
    config = tomli.load(f)

Typed Configuration Classes

Using Dataclasses

import os
from dataclasses import dataclass, field
from typing import List
from pathlib import Path

@dataclass
class DatabaseConfig:
    url: str = "sqlite:///dev.db"
    pool_size: int = 10
    timeout: float = 30.0

    def is_sqlite(self) -> bool:
        return self.url.startswith("sqlite")

@dataclass
class ServerConfig:
    host: str = "0.0.0.0"
    port: int = 8000
    debug: bool = False
    workers: int = 4

@dataclass
class RedisConfig:
    url: str = "redis://localhost:6379"
    ttl: int = 300

@dataclass
class Settings:
    database: DatabaseConfig = field(default_factory=DatabaseConfig)
    server: ServerConfig = field(default_factory=ServerConfig)
    redis: RedisConfig = field(default_factory=RedisConfig)
    secret_key: str = ""
    allowed_origins: List[str] = field(default_factory=lambda: ["http://localhost:3000"])

    @classmethod
    def from_env(cls) -> "Settings":
        return cls(
            database=DatabaseConfig(
                url=os.environ.get("DATABASE_URL", "sqlite:///dev.db"),
                pool_size=int(os.environ.get("DB_POOL_SIZE", "10")),
                timeout=float(os.environ.get("DB_TIMEOUT", "30.0")),
            ),
            server=ServerConfig(
                host=os.environ.get("HOST", "0.0.0.0"),
                port=int(os.environ.get("PORT", "8000")),
                debug=os.environ.get("DEBUG", "false").lower() == "true",
                workers=int(os.environ.get("WORKERS", "4")),
            ),
            redis=RedisConfig(
                url=os.environ.get("REDIS_URL", "redis://localhost:6379"),
                ttl=int(os.environ.get("REDIS_TTL", "300")),
            ),
            secret_key=os.environ.get("SECRET_KEY", ""),
            allowed_origins=os.environ.get("ALLOWED_ORIGINS", "").split(","),
        )

# Usage
settings = Settings.from_env()
print(settings.database.url)
print(settings.server.port)
print(settings.server.debug)

Using Pydantic Settings (Recommended)

# pip install pydantic pydantic-settings
from pydantic_settings import BaseSettings
from pydantic import Field, field_validator
from typing import List
from enum import Enum

class Environment(str, Enum):
    development = "development"
    staging = "staging"
    production = "production"

class DatabaseSettings(BaseSettings):
    url: str = "sqlite:///dev.db"
    pool_size: int = 10
    timeout: float = 30.0

    model_config = {"env_prefix": "DB_"}

class Settings(BaseSettings):
    model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}

    # Required (no default)
    secret_key: str

    # Optional with defaults
    environment: Environment = Environment.development
    debug: bool = False
    host: str = "0.0.0.0"
    port: int = 8000
    allowed_origins: List[str] = ["http://localhost:3000"]
    max_connections: int = 10
    timeout: float = 30.0
    database_url: str = "sqlite:///dev.db"

    @field_validator("allowed_origins", mode="before")
    @classmethod
    def parse_origins(cls, v):
        if isinstance(v, str):
            return [origin.strip() for origin in v.split(",")]
        return v

    @field_validator("debug")
    @classmethod
    def set_debug_from_env(cls, v, info):
        if info.data.get("environment") == Environment.production:
            return False
        return v

    def is_production(self) -> bool:
        return self.environment == Environment.production

# Usage
settings = Settings()  # Automatically reads from environment
print(settings.database_url)
print(settings.is_production())

Environment-Specific Configuration

import os
from pathlib import Path
from dotenv import load_dotenv

class ConfigManager:
    def __init__(self):
        self.env = os.environ.get("APP_ENV", "development")
        self._load_env_file()

    def _load_env_file(self):
        env_files = {
            "development": ".env",
            "staging": ".env.staging",
            "production": ".env.production",
        }
        env_file = env_files.get(self.env, ".env")
        if Path(env_file).exists():
            load_dotenv(env_file)

    @property
    def database_url(self):
        return os.environ.get("DATABASE_URL", "sqlite:///dev.db")

    @property
    def debug(self):
        return self.env != "production"

    @property
    def log_level(self):
        levels = {
            "development": "DEBUG",
            "staging": "INFO",
            "production": "WARNING",
        }
        return levels.get(self.env, "INFO")

    @property
    def allowed_hosts(self):
        if self.env == "production":
            return os.environ.get("ALLOWED_HOSTS", "").split(",")
        return ["*"]

# Usage
config = ConfigManager()
print(f"Environment: {config.env}")
print(f"Debug: {config.debug}")
print(f"Log Level: {config.log_level}")

12-Factor App Configuration Principles

Factor 3: Config
Store config in the environment.

The twelve-factor app stores config in environment variables.

What belongs in config?
  - Database credentials and URLs
  - API keys and secrets
  - Feature flags
  - Port numbers
  - Log levels
  - Service URLs

What does NOT belong in config?
  - Code
  - Business logic
  - Static assets

Configuration Hierarchy

Priority (highest to lowest):
1. Command-line arguments
2. Environment variables
3. .env files
4. Config files (YAML, JSON, TOML)
5. Default values in code

Common Mistakes

Best Practices

1. Never hardcode configuration values in source code
2. Use .env files for local development (add to .gitignore!)
3. Use environment variables in production
4. Create typed config classes for validation and IDE support
5. Validate required configuration at startup, not at runtime
6. Keep secrets out of version control — use secret managers
7. Use different configs for dev/staging/production
8. Document required environment variables in README
9. Use Pydantic Settings for automatic validation and type conversion
10. Fail fast — raise errors immediately if required config is missing

Key Takeaways

1. Never hardcode configuration values in source code
2. Use .env files for local development (add to .gitignore!)
3. Use environment variables in production
4. Create typed config classes for validation and IDE support
5. Validate required configuration at startup
6. Keep secrets out of version control
7. Use different configs for dev/staging/production
8. Document required environment variables in README
9. Use Pydantic Settings for automatic validation and type conversion
10. Follow 12-factor app principles for cloud-native applications