ML Credit Risk Scoring with Fairness

What is Credit Risk Scoring?

Credit risk scoring estimates the probability that a borrower will default on their debt obligations within a specified time horizon (typically 12 months). Lenders use these scores to price loans, set credit limits, and make approve/decline decisions. The global credit scoring market exceeds $30 billion, with FICO scores alone used by 90% of US lending decisions.

Traditional credit scoring relied on logistic regression with manually selected features based on expert judgment. The FICO score model uses five factors: payment history (35%), credit utilization (30%), length of credit history (15%), credit mix (10%), and new credit inquiries (10%). This model has remained largely unchanged since 1989, despite dramatic increases in available data and modeling techniques.

Modern ML credit scoring incorporates thousands of alternative data points — utility payments, rent history, mobile phone usage, social media activity, and transaction patterns. These features improve prediction accuracy by 5–15% over traditional models, but introduce significant fairness concerns. Protected attributes (race, gender, age) may correlate with legitimate features, creating disparate impact even when protected attributes aren't explicitly used.

Regulatory compliance is non-negotiable. The Equal Credit Opportunity Act (ECOA) requires lenders to provide specific adverse action reasons when declining applicants. The Fair Credit Reporting Act (FCRA) mandates accuracy in credit reports. Model risk management guidelines (SR 11-7) require comprehensive documentation, independent validation, and ongoing monitoring. A model that predicts well but cannot explain its decisions to regulators cannot be deployed.

Project Architecture

Tools & Setup

Step 1: Environment Setup

pip install scikit-learn xgboost shap pandas numpy matplotlib lifelines

Step 2: Data Loading

Mathematical Foundation

Logistic Regression (primary model for interpretability):

Where:

• — coefficient for feature
• — standardized feature value
• Intuition: Each feature shifts the log-odds of default linearly

Weight of Evidence (WOE) encoding:

Where:

• — proportion of defaults in bin
• Intuition: Measures the predictive power of each feature bin

Gini Coefficient (model discrimination):

Where:

• AUC — Area under ROC curve
• Intuition: Probability that a random defaulter scores lower than a random non-defaulter

Disparate Impact Ratio (fairness):

Where:

• — approved (non-default prediction)
• Intuition: Ratio of approval rates; must exceed 0.8 under EEOC guidelines

Model Architecture — Logistic Regression

SHAP Explainability

Fairness Evaluation

Performance Results

Real-World Case Study

Upstart, an AI-first lender, uses ML models incorporating education and employment data to expand credit access. Their models approve 27% more borrowers with 16% lower average APRs than traditional models. Key operational metrics: $30B+ in loans originated, 74% approval rate (vs. 55% for traditional models), average FICO of 650 (vs. 700 for bank-originated loans). Their explainability system generates specific adverse action reasons for each decline, maintaining ECOA compliance while using 1,000+ features.

Deployment

Common Pitfalls

1. Data leakage: Using features that wouldn't be available at application time (e.g., payment history for new applicants)
2. Survivorship bias: Only modeling applicants who were previously approved ignores the rejected population
3. Fairness-accuracy tradeoff: Strict demographic parity constraints can reduce model performance by 5–10%
4. Calibration issues: Uncalibrated probabilities lead to mispriced loans — always use isotonic regression
5. Concept drift: Economic conditions change default rates — models need quarterly recalibration

Summary with Key Takeaways

This project built a regulatory-compliant credit scoring system achieving 0.782 AUC with logistic regression, maintaining disparate impact ratio above 0.80. The SHAP-based explainability system generates specific adverse action reasons required by ECOA. Key principles: interpretable models (logistic regression) are preferred for regulated lending despite lower performance; calibration is essential for accurate PD estimation; and fairness testing must be integrated into the development pipeline, not bolted on afterward.