Python Performance — Profiling & Optimization

Understanding performance helps you write faster, more efficient Python code.

Learning Objectives

• Profile code to find bottlenecks
• Use timeit for micro-benchmarks
• Apply common performance optimizations
• Use memory profiling tools

timeit — Micro-benchmarks

import timeit

# Time a statement
time = timeit.timeit('sum(range(1000))', number=10000)
print(f"Total: {time:.4f}s")

# Time a function
def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

time = timeit.timeit(lambda: fib(30), number=10)
print(f"Fib(30) x10: {time:.4f}s")

cProfile — Profiling

import cProfile

def slow_function():
    total = 0
    for i in range(1000000):
        total += i
    return total

cProfile.run('slow_function()')

Common Optimizations

# Use sets for membership testing
bad = [1, 2, 3, 4, 5] * 10000
good = set(bad)
99999 in bad   # Slow O(n)
99999 in good  # Fast O(1)

# Use generators for large datasets
total = sum(x**2 for x in range(1000000))

# Use list comprehensions over loops
result = [i * 2 for i in range(1000)]

# Use join for string concatenation
result = "".join(words)

# Use built-in functions
total = sum(numbers)
minimum = min(numbers)
sorted_list = sorted(numbers)

Memory Optimization

import sys

# Check object size
print(sys.getsizeof(42))           # 28 bytes
print(sys.getsizeof("hello"))      # 54 bytes
print(sys.getsizeof([1, 2, 3]))    # 88 bytes

# Use __slots__ to reduce class memory
class Point:
    __slots__ = ('x', 'y')
    def __init__(self, x, y):
        self.x = x
        self.y = y

Key Takeaways

1. Profile first — don't optimize blindly
2. Use sets for O(1) membership testing
3. Use generators for memory-efficient iteration
4. Use list comprehensions over manual loops
5. Use __slots__ to reduce class memory overhead