Scipy Optimization

Introduction

SciPy provides powerful optimization tools in the scipy.optimize module for finding minima, maxima, and roots of functions.

Minimization

from scipy.optimize import minimize

# Unconstrained minimization
def rosen(x):
    return sum(100*(x[1:]-x[:-1]**2)**2 + (1-x[:-1])**2)

result = minimize(rosen, x0=[1, 2, 3, 4, 5], method='Nelder-Mead')
print(result.x)  # Optimal solution
print(result.fun)  # Objective value

Root Finding

from scipy.optimize import root, fsolve

def equations(p):
    x, y = p
    return [x**2 + y**2 - 1, y - x**2]

sol = root(equations, [0.5, 0.5])
print(sol.x)  # Solution roots

# Alternative using fsolve
sol2 = fsolve(equations, [0.5, 0.5])
print(sol2)

Constrained Optimization

from scipy.optimize import minimize

# With bounds
result = minimize(lambda x: x**2, x0=[5], bounds=[(0, None)])

# With constraints
from scipy.optimize import LinearConstraint
A = [[1, 1]]
lb = [1]
ub = [2]
constraint = LinearConstraint(A, lb, ub)
result = minimize(lambda x: x[0]**2 + x[1]**2, [0, 0], constraints=constraint)

Curve Fitting

from scipy.optimize import curve_fit

def model(x, a, b, c):
    return a * np.exp(-b * x) + c

x_data = np.array([0, 1, 2, 3, 4])
y_data = np.array([2.1, 1.2, 0.8, 0.5, 0.4])

popt, pcov = curve_fit(model, x_data, y_data, p0=[2, 0.5, 0])
print(popt)  # Fitted parameters

Practice Problems

1. Find the minimum of a multi-dimensional function using different methods
2. Solve a system of nonlinear equations
3. Fit a polynomial to data using curve_fit
4. Implement bounded optimization with constraints
5. Compare BFGS vs Nelder-Mead convergence