Type Hints: Generic, Protocol, TypeVar, overload, Literal
Advanced type hints for better code quality and documentation
Interview Question
"Explain advanced Python type hints. How do Generic types, Protocol, TypeVar, overload, and Literal work? When would you use each? How do type hints improve code quality?"
Difficulty: Medium | Frequently asked at Google, Meta, Amazon
Theoretical Foundation
Why Type Hints?
# Without type hints - unclear API
def process(data, config):
pass
# With type hints - clear API
def process(data: List[Dict[str, Any]], config: ProcessConfig) -> Result:
pass
# Benefits:
# 1. Better IDE autocompletion
# 2. Static type checking (mypy, pyright)
# 3. Self-documenting code
# 4. Catch errors early
ℹ️
Key Concept: Type hints don't enforce types at runtime but enable static analysis tools to catch bugs.
Basic Type Hints
from typing import (
List, Dict, Tuple, Set, Optional, Union,
Any, Callable, Type, Sequence, Mapping
)
# Basic types
name: str = "Alice"
age: int = 30
height: float = 5.9
is_active: bool = True
# Collections
numbers: List[int] = [1, 2, 3]
user: Dict[str, Any] = {"name": "Alice", "age": 30}
coordinates: Tuple[float, float] = (40.7128, -74.0060)
unique_ids: Set[int] = {1, 2, 3}
# Optional and Union
def greet(name: Optional[str] = None) -> str:
return f"Hello, {name or 'World'}!"
def process(value: Union[str, int]) -> str:
return str(value)
# Callable
def apply_func(func: Callable[[int], int], value: int) -> int:
return func(value)
# Type alias
UserId = int
UserName = str
UserDict = Dict[UserId, UserName]
TypeVar
Basic TypeVar
from typing import TypeVar, List
# Define type variable
T = TypeVar('T')
Numeric = TypeVar('Numeric', int, float)
def first_element(lst: List[T]) -> T:
"""Return first element of list."""
return lst[0]
def add(a: Numeric, b: Numeric) -> Numeric:
"""Add two numeric values."""
return a + b
# Usage - type is inferred
result1 = first_element([1, 2, 3]) # Type: int
result2 = first_element(["a", "b"]) # Type: str
# TypeVar with constraints
SortKey = TypeVar('SortKey')
def sort_by_key(items: List[T], key: Callable[[T], SortKey]) -> List[T]:
"""Sort items by key function."""
return sorted(items, key=key)
# TypeVar with bounds
from typing import Comparable
Orderable = TypeVar('Orderable')
def find_max(items: List[Orderable]) -> Orderable:
"""Find maximum item."""
return max(items)
Constrained TypeVar
from typing import TypeVar, List
# Constrained to specific types
Numeric = TypeVar('Numeric', int, float, complex)
def multiply(a: Numeric, b: Numeric) -> Numeric:
return a * b
# Works with int
result1 = multiply(2, 3) # OK
# Works with float
result2 = multiply(2.5, 3.5) # OK
# Would fail type checking
# result3 = multiply("2", "3") # Error!
# TypeVar with bounds
from typing import Any
Comparable = TypeVar('Comparable')
def max_value(a: Comparable, b: Comparable) -> Comparable:
return a if a > b else b
# TypeVar for return type
T = TypeVar('T')
def identity(value: T) -> T:
return value
# Type is preserved
x = identity(42) # Type: int
y = identity("hello") # Type: str
💡
Interview Tip: TypeVar allows you to write generic functions that preserve type information.
Generic Types
Generic Classes
from typing import TypeVar, Generic, List, Optional
T = TypeVar('T')
K = TypeVar('K')
V = TypeVar('V')
# Generic stack
class Stack(Generic[T]):
def __init__(self) -> None:
self._items: List[T] = []
def push(self, item: T) -> None:
self._items.append(item)
def pop(self) -> T:
if not self._items:
raise IndexError("Stack is empty")
return self._items.pop()
def peek(self) -> T:
if not self._items:
raise IndexError("Stack is empty")
return self._items[-1]
def is_empty(self) -> bool:
return len(self._items) == 0
def size(self) -> int:
return len(self._items)
# Usage
int_stack: Stack[int] = Stack()
int_stack.push(1)
int_stack.push(2)
value: int = int_stack.pop() # Type is preserved
str_stack: Stack[str] = Stack()
str_stack.push("hello")
str_value: str = str_stack.pop()
# Generic mapping
class GenericDict(Generic[K, V]):
def __init__(self) -> None:
self._dict: Dict[K, V] = {}
def set(self, key: K, value: V) -> None:
self._dict[key] = value
def get(self, key: K) -> Optional[V]:
return self._dict.get(key)
# Usage
user_dict: GenericDict[int, str] = GenericDict()
user_dict.set(1, "Alice")
name: Optional[str] = user_dict.get(1)
Generic with Multiple Type Parameters
from typing import TypeVar, Generic, List, Tuple, Optional
T = TypeVar('T')
U = TypeVar('U')
class Pair(Generic[T, U]):
def __init__(self, first: T, second: U) -> None:
self.first = first
self.second = second
def swap(self) -> 'Pair[U, T]':
return Pair(self.second, self.first)
def to_tuple(self) -> Tuple[T, U]:
return (self.first, self.second)
# Usage
pair: Pair[int, str] = Pair(1, "hello")
swapped: Pair[str, int] = pair.swap()
t: Tuple[int, str] = pair.to_tuple()
# Generic container
class Container(Generic[T]):
def __init__(self, items: List[T]) -> None:
self.items = items
def filter(self, predicate: Callable[[T], bool]) -> 'Container[T]':
return Container([item for item in self.items if predicate(item)])
def map(self, func: Callable[[T], U]) -> 'Container[U]':
return Container([func(item) for item in self.items])
# Usage
numbers = Container([1, 2, 3, 4, 5])
even = numbers.filter(lambda x: x % 2 == 0)
doubled = numbers.map(lambda x: x * 2)
Protocol (Structural Subtyping)
Basic Protocol
from typing import Protocol, runtime_checkable
# Protocol defines interface without inheritance
@runtime_checkable
class Drawable(Protocol):
def draw(self) -> str: ...
@runtime_checkable
class Resizable(Protocol):
def resize(self, factor: float) -> None: ...
# Classes that satisfy protocol (duck typing)
class Circle:
def __init__(self, radius: float):
self.radius = radius
def draw(self) -> str:
return f"Drawing circle with radius {self.radius}"
def resize(self, factor: float) -> None:
self.radius *= factor
class Square:
def __init__(self, side: float):
self.side = side
def draw(self) -> str:
return f"Drawing square with side {self.side}"
def resize(self, factor: float) -> None:
self.side *= factor
# Functions that accept protocols
def draw_shape(shape: Drawable) -> str:
return shape.draw()
def resize_shape(shape: Resizable, factor: float) -> None:
shape.resize(factor)
# Usage
circle = Circle(5)
square = Square(3)
print(draw_shape(circle)) # Works
print(draw_shape(square)) # Works
resize_shape(circle, 2) # Works
resize_shape(square, 2) # Works
# Runtime check
print(isinstance(circle, Drawable)) # True
print(isinstance(circle, Resizable)) # True
Advanced Protocol
from typing import Protocol, TypeVar, List, Optional
T = TypeVar('T')
# Protocol with generic
class Container(Protocol[T]):
def add(self, item: T) -> None: ...
def get(self, index: int) -> T: ...
def size(self) -> int: ...
# Protocol with multiple methods
class Serializable(Protocol):
def to_dict(self) -> dict: ...
@classmethod
def from_dict(cls, data: dict) -> 'Serializable': ...
# Protocol inheritance
class Readable(Protocol):
def read(self) -> str: ...
class Writable(Protocol):
def write(self, data: str) -> None: ...
class ReadWriteable(Readable, Writable, Protocol):
pass
# Concrete implementation
class File:
def __init__(self, name: str):
self.name = name
self.content = ""
def read(self) -> str:
return self.content
def write(self, data: str) -> None:
self.content += data
# Function accepting protocol
def process_readable(reader: Readable) -> str:
return reader.read()
def process_writable(writer: Writable, data: str) -> None:
writer.write(data)
# Usage
file = File("test.txt")
print(process_readable(file))
process_writable(file, "Hello")
ℹ️
Protocol vs ABC: Protocol uses structural subtyping (duck typing), ABC uses nominal subtyping (inheritance).
overload
Basic overload
from typing import overload, Union, List
# Overload for different argument types
@overload
def process(value: int) -> int: ...
@overload
def process(value: str) -> str: ...
@overload
def process(value: List[int]) -> int: ...
def process(value: Union[int, str, List[int]]) -> Union[int, str]:
if isinstance(value, int):
return value * 2
elif isinstance(value, str):
return value.upper()
else:
return sum(value)
# Usage - type checker knows return type
result1: int = process(5) # Returns int
result2: str = process("hello") # Returns str
result3: int = process([1, 2, 3]) # Returns int
Advanced overload
from typing import overload, Union, Optional, List
class DataProcessor:
@overload
def process(self, data: int) -> int: ...
@overload
def process(self, data: str) -> str: ...
@overload
def process(self, data: List[int]) -> List[int]: ...
@overload
def process(self, data: List[str]) -> List[str]: ...
def process(self, data: Union[int, str, List[int], List[str]]) -> Union[int, str, List[int], List[str]]:
if isinstance(data, int):
return data * 2
elif isinstance(data, str):
return data.upper()
elif isinstance(data, list):
if data and isinstance(data[0], int):
return [x * 2 for x in data]
else:
return [x.upper() for x in data]
return data
# Usage
processor = DataProcessor()
r1: int = processor.process(5)
r2: str = processor.process("hello")
r3: List[int] = processor.process([1, 2, 3])
r4: List[str] = processor.process(["a", "b", "c"])
# Type checker knows exact return type based on input
Overload with None
from typing import overload, Optional
class UserService:
@overload
def get_user(self, user_id: int) -> dict: ...
@overload
def get_user(self, user_id: None) -> None: ...
def get_user(self, user_id: Optional[int]) -> Optional[dict]:
if user_id is None:
return None
return {"id": user_id, "name": f"User {user_id}"}
# Usage
service = UserService()
user: dict = service.get_user(1) # Returns dict
nothing: None = service.get_user(None) # Returns None
⚠️
Note: Overload is for type checkers only. The implementation must handle all cases.
Literal
Basic Literal
from typing import Literal
# Literal types
Direction = Literal["north", "south", "east", "west"]
HttpStatus = Literal[200, 301, 404, 500]
BooleanLike = Literal[True, False, "yes", "no"]
def move(direction: Direction) -> str:
return f"Moving {direction}"
def get_status(code: HttpStatus) -> str:
messages = {200: "OK", 301: "Moved", 404: "Not Found", 500: "Error"}
return messages.get(code, "Unknown")
# Usage - type checker validates
move("north") # OK
move("south") # OK
# move("up") # Error!
get_status(200) # OK
get_status(404) # OK
# get_status(201) # Error!
Literal with Union
from typing import Literal, Union
# Complex literal types
LogLevel = Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
FileType = Literal["csv", "json", "xml", "yaml"]
def log_message(level: LogLevel, message: str) -> None:
print(f"[{level}] {message}")
def process_file(file_type: FileType, content: str) -> str:
if file_type == "csv":
return f"Processing CSV: {content}"
elif file_type == "json":
return f"Processing JSON: {content}"
else:
return f"Processing {file_type}: {content}"
# Literal in dataclass
from dataclasses import dataclass
@dataclass
class Config:
mode: Literal["development", "production", "testing"]
log_level: LogLevel
file_type: FileType
# Usage
config = Config(mode="production", log_level="INFO", file_type="json")
Literal with enums
from typing import Literal
from enum import Enum
class Color(Enum):
RED = "red"
GREEN = "green"
BLUE = "blue"
# Literal can work with enum values
def paint(color: Literal["red", "green", "blue"]) -> str:
return f"Painting {color}"
# Or use enum directly
def paint_enum(color: Color) -> str:
return f"Painting {color.value}"
# Usage
paint("red") # OK
paint_enum(Color.RED) # OK
💡
Interview Tip: Literal types enable type checkers to validate specific values, catching errors at compile time.
Advanced Patterns
TypedDict
from typing import TypedDict, List, Optional
# TypedDict for dictionary structure
class UserDict(TypedDict):
id: int
name: str
email: str
age: Optional[int]
class PostDict(TypedDict):
id: int
title: str
content: str
author: UserDict
tags: List[str]
# Usage
user: UserDict = {
"id": 1,
"name": "Alice",
"email": "alice@example.com",
"age": 30
}
post: PostDict = {
"id": 1,
"title": "Hello World",
"content": "This is a post",
"author": user,
"tags": ["python", "typing"]
}
# TypedDict with optional keys
class ConfigDict(TypedDict, total=False):
host: str
port: int
debug: bool
# Required keys
class RequiredConfig(TypedDict):
host: str
port: int
class FullConfig(RequiredConfig, total=False):
debug: bool
log_level: str
NewType
from typing import NewType
# Create distinct types
UserId = NewType('UserId', int)
OrderId = NewType('OrderId', int)
Email = NewType('Email', str)
# Functions with distinct types
def get_user(user_id: UserId) -> dict:
return {"id": user_id, "name": "Alice"}
def get_order(order_id: OrderId) -> dict:
return {"id": order_id, "total": 100}
def send_email(email: Email, subject: str) -> bool:
print(f"Sending to {email}: {subject}")
return True
# Usage - type checker catches errors
user_id = UserId(1)
order_id = OrderId(100)
get_user(user_id) # OK
get_order(order_id) # OK
# These would fail type checking:
# get_user(order_id) # Error! Wrong type
# get_order(user_id) # Error! Wrong type
# NewType is erased at runtime
print(type(user_id)) # <class 'int'>
Final and ClassVar
from typing import Final, ClassVar
from dataclasses import dataclass
# Final - cannot be reassigned
MAX_SIZE: Final = 100
PI: Final = 3.14159
# Would fail:
# MAX_SIZE = 200 # Error!
@dataclass
class Constants:
# ClassVar - not included in __init__
class_count: ClassVar[int] = 0
# Final - cannot be reassigned
instance_id: Final[int]
name: str
def __post_init__(self):
Constants.class_count += 1
# Usage
c1 = Constants(instance_id=1, name="First")
c2 = Constants(instance_id=2, name="Second")
print(Constants.class_count) # 2
# c1.instance_id = 3 # Error! Final
Type Checking with mypy
Configuration
# mypy.ini
[mypy]
python_version = 3.9
warn_return_any = True
warn_unused_configs = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
check_untyped_defs = True
disallow_untyped_decorators = True
no_implicit_optional = True
warn_redundant_casts = True
warn_unused_ignores = True
warn_return_any = True
no_implicit_reexport = True
strict_equality = True
# Per-module overrides
[mypy-third_party.*]
ignore_missing_imports = True
[mypy-tests.*]
disallow_untyped_defs = False
Type Checking Examples
from typing import List, Dict, Any, Optional
# Strict typing
def process_data(data: List[Dict[str, Any]]) -> Dict[str, List[Any]]:
result: Dict[str, List[Any]] = {}
for item in data:
for key, value in item.items():
if key not in result:
result[key] = []
result[key].append(value)
return result
# With type ignore (use sparingly)
def legacy_function() -> None:
# type: ignore[no-untyped-def]
pass
# Cast for complex situations
from typing import cast
def complex_operation(value: Any) -> str:
# Cast when you know the type
result = cast(str, value)
return result.upper()
ℹ️
Type Checking Tip: Run mypy in CI/CD to catch type errors before they reach production.
Complexity Analysis
Type Hint Overhead
| Feature | Runtime Cost | Type Checker Cost | Use Case |
|---|---|---|---|
| Basic hints | None | Low | All code |
| TypeVar | None | Medium | Generic code |
| Protocol | None | Medium | Duck typing |
| overload | None | High | Multiple signatures |
| Literal | None | Low | Specific values |
Memory Impact
import sys
from typing import List, Dict
# Type hints don't affect runtime memory
def with_hints(data: List[int]) -> Dict[str, int]:
return {"sum": sum(data)}
def without_hints(data):
return {"sum": sum(data)}
# Both have same memory footprint
# Type hints are erased at runtime
Interview Tips
Common Follow-up Questions
-
"When should you use type hints?"
- Public APIs
- Complex functions
- Library code
- Team projects
-
"What's the difference between TypeVar and Generic?"
- TypeVar: Defines type variable
- Generic: Uses type variable in class
-
"Protocol vs ABC - when to use which?"
- Protocol: Structural subtyping, duck typing
- ABC: Nominal subtyping, explicit inheritance
Code Review Tips
# BAD: Using Any too much
def process(data: Any) -> Any:
return data
# GOOD: Specific types
def process(data: List[Dict[str, int]]) -> Dict[str, int]:
return {"sum": sum(item.get("value", 0) for item in data)}
# BAD: Missing type hints
def calculate(a, b):
return a + b
# GOOD: Complete type hints
def calculate(a: Union[int, float], b: Union[int, float]) -> Union[int, float]:
return a + b
# BAD: Overusing overload
@overdef process(x: int) -> int: ...
@overdef process(x: str) -> str: ...
def process(x: Union[int, str]) -> Union[int, str]:
return x
# GOOD: Simple union when appropriate
def process(x: Union[int, str]) -> Union[int, str]:
return x
⚠️
Common Mistake: Adding type hints to every single line. Focus on public APIs and complex functions.
Summary
| Feature | Purpose | When to Use |
|---|---|---|
| TypeVar | Generic types | Generic functions/classes |
| Protocol | Structural subtyping | Duck typing |
| overload | Multiple signatures | Different arg types |
| Literal | Specific values | Constants, enums |
| TypedDict | Dict structure | JSON-like data |
| NewType | Distinct types | Type safety |
Best Practices
- Start simple - basic hints first
- Use TypeVar for generic code
- Use Protocol for duck typing
- Use overload sparingly
- Run mypy in CI/CD
- Document complex types
ℹ️
Key Takeaway: Type hints improve code quality, enable better tooling, and serve as documentation.
Practice Problems
- Generic Container: Implement a generic stack with TypeVar
- Protocol Design: Create protocols for a file system
- Overload Functions: Write overloaded functions for different types
- Literal Types: Design a configuration system with Literal types
- TypedDict: Create TypedDict for API responses
Further Reading
- PEP 484: Type Hints
- PEP 544: Protocols
- PEP 586: Literal Types
- mypy Documentation: https://mypy.readthedocs.io/
Remember: Type hints are a tool for better code quality. Use them judiciously and consistently.