Timeline of GPT Models:
-------------------------------------------------------------
2018  GPT-1    117M params   First generative pre-training
2019  GPT-2    1.5B params   "Too dangerous to release"
2020  GPT-3    175B params   In-context learning revolution
2022  ChatGPT  GPT-3.5      RLHF alignment breakthrough
2023  GPT-4    ~1.8T MoE     Multimodal, reasoning leap
2024  GPT-4o   ~1.8T MoE     Omni-modal, 2x speed
2024  o1/o3    Unknown       Chain-of-thought reasoning
2025  GPT-4.1  Unknown       1M context, latest knowledge
-------------------------------------------------------------

P(token_t | token_1, token_2, ..., token_{t-1})

Example:
Input:  "The capital of France is"
Output: "Paris" (with highest probability)

Internally:
token_1 = "The"      -> logits for all 100K tokens
token_2 = "capital"  -> logits updated
token_3 = "of"       -> logits updated
token_4 = "France"   -> logits updated
token_5 = "is"       -> logits updated
token_6 = ?          -> "Paris" has highest probability

Self-Attention Formula:

Attention(Q, K, V) = softmax(QK^T / √d_k) V

Where:
Q = Query matrix   (what am I looking for?)
K = Key matrix     (what do I contain?)
V = Value matrix   (what information do I provide?)
d_k = Key dimension (for scaling)

Intuition:
Each token creates a Query ("What context do I need?")
Each token creates a Key   ("What context can I provide?")
Each token creates a Value ("What information to pass on")

Query × Key = Attention scores (how much to attend to each token)
Attention scores × Value = Output (weighted combination of information)

Multi-Head Attention:

Head 1: Attention(Q₁, K₁, V₁) -> learns syntax
Head 2: Attention(Q₂, K₂, V₂) -> learns semantics
Head 3: Attention(Q₃, K₃, V₃) -> learns long-range deps
...
Head h: Attention(Qₕ, Kₕ, Vₕ) -> learns other patterns

Output = Concat(head₁, ..., headₕ) × W^O

GPT-4 uses 128 attention heads per layer (estimated)

+---------------------------------------------+
|           Transformer Block                  |
|                                              |
|  Input Embeddings                            |
|       |                                      |
|       v                                      |
|  +---------------------------------+        |
|  |  Multi-Head Self-Attention      |        |
|  |  + Residual Connection          |        |
|  |  + Layer Normalization          |        |
|  +---------------+-----------------+        |
|                  |                           |
|                  v                           |
|  +---------------------------------+        |
|  |  Feed-Forward Network           |        |
|  |  (2 × d_model internal dim)    |        |
|  |  + Residual Connection          |        |
|  |  + Layer Normalization          |        |
|  +---------------+-----------------+        |
|                  |                           |
|                  v                           |
|  Output Embeddings                           |
+---------------------------------------------+

Stacked N times:
GPT-3:       96 layers
GPT-4:       ~120 layers (estimated)
GPT-4o:      ~120 layers (estimated)

GPT uses learned positional embeddings:

Token Embedding + Position Embedding = Input

"The"    at position 0 -> embedding + pos_0
"cat"    at position 1 -> embedding + pos_1
"sat"    at position 2 -> embedding + pos_2

GPT-4 uses ALiBi (Attention with Linear Biases) for
better extrapolation to longer sequences than seen in training.

In-Context Learning:

Few-shot prompt:
"Convert English to French:
  hello -> bonjour
  goodbye -> au revoir
  thank you ->"

GPT-3 output: "merci"

No fine-tuning needed! Just provide examples in the prompt.

Dense Model (GPT-3):
Every parameter is used for every token
175B params -> 175B active per token
Slow but thorough

Mixture of Experts (GPT-4o):
Multiple "expert" networks, only some activated per token
~1.8T total params -> ~220B active per token
Fast AND capable

+---------------------------------------------+
|           MoE Architecture                   |
|                                              |
|  Input Token                                 |
|       |                                      |
|       v                                      |
|  +---------------------+                    |
|  |  Gating Network      | <- Learns which    |
|  |  (Router)            |   experts to use  |
|  +----------+----------+                    |
|             |                                |
|    +--------+--------+--------+             |
|    v        v        v        v             |
|  +-----+ +-----+ +-----+ +-----+           |
|  |Exp 1| |Exp 2| |Exp 3| |...  | (8 total) |
|  |220B | |220B | |220B | |Exp 8|           |
|  +--+--+ +--+--+ +--+--+ +--+--+           |
|     +--------+--------+     |               |
|              v              |               |
|  +---------------------+   |               |
|  |  Top-2 Selection     |◄--+               |
|  |  (Only 2 experts    |                    |
|  |   active per token) |                    |
|  +----------+----------+                    |
|             v                                |
|  Combined Output                             |
+---------------------------------------------+

Benefits:
- Same quality as 1.8T dense model
- Runs at ~220B speed (8x faster)
- Lower compute cost
- Better scaling properties

Previous Approach (GPT-4 + Whisper + DALL-E):
Audio -> Whisper (STT) -> Text -> GPT-4 -> Text -> TTS -> Audio
(Latency: ~5 seconds round-trip)

GPT-4o Native Approach:
Audio -> [GPT-4o Unified Model] -> Audio
(Latency: ~300ms round-trip, 16x faster!)

The model processes:
- Text tokens (via text tokenizer)
- Image patches (via vision encoder)
- Audio spectrograms (via audio encoder)

All in the SAME transformer layers!

Standard LLM (System 1 Thinking):
Input -> Immediate Output
"7 × 8 = ?" -> "54" (often wrong)

Reasoning Model (System 2 Thinking):
Input -> Think -> Think -> Think -> Verify -> Output
"7 × 8 = ?"
Step 1: "7 × 8... let me break this down"
Step 2: "7 × 8 = 7 × (10 - 2) = 70 - 14"
Step 3: "70 - 14 = 56"
Step 4: "Let me verify: 8 × 7 = 56 ✓"
Output: "56"

Reasoning Model Internal Process:

1. Problem Decomposition
   "Solve x² + 5x + 6 = 0"
   -> "This is a quadratic equation"
   -> "I can factor this"

2. Step-by-Step Reasoning
   "x² + 5x + 6 = (x + 2)(x + 3) = 0"
   "Therefore x = -2 or x = -3"

3. Self-Verification
   "Check: (-2)² + 5(-2) + 6 = 4 - 10 + 6 = 0 ✓"
   "Check: (-3)² + 5(-3) + 6 = 9 - 15 + 6 = 0 ✓"

4. Final Answer
   "x = -2 or x = -3"

This "thinking" happens invisibly — the user only sees the final answer.

Objective: Predict next token on massive text corpus

Data: ~13 trillion tokens from:
- Web pages (Common Crawl)
- Books (Books3, Gutenberg)
- Code (GitHub, Stack Overflow)
- Academic papers (ArXiv, Semantic Scholar)
- Wikipedia
- News articles

Loss function:
L = -Σ log P(token_t | token_1, ..., token_{t-1})

Training compute:
GPT-3:     ~3.14 × 10²³ FLOPs
GPT-4:     ~2.1 × 10²⁵ FLOPs (estimated)
GPT-4o:    ~2.6 × 10²⁵ FLOPs (estimated)

Human-written examples teach the model to follow instructions:

Prompt: "Explain quantum computing in simple terms"
Ideal Response: [Human-written explanation]

Thousands of these examples fine-tune the pre-trained model
to behave as a helpful assistant.

RLHF Process:

Step 1: Collect Comparisons
Human sees: "What is 2+2?"
Model generates 4 different responses
Human ranks: Response 3 > Response 1 > Response 4 > Response 2

Step 2: Train Reward Model
Learn to predict human preference scores
R(response | prompt) -> score

Step 3: Optimize with PPO
Maximize: E[R(response)] - β × KL(response || reference)
- Maximize reward (human preference)
- Stay close to original model (KL penalty)

Result: Model learns to generate responses humans prefer

Constitutional AI principles:
1. Be helpful and harmless
2. Be honest and accurate
3. Respect human autonomy
4. Avoid bias and discrimination
5. Protect privacy and security

The model self-critiques against these principles
and revises its responses accordingly.

Strategy 1: Model Routing
Simple query -> GPT-4o mini ($0.15/M)
Complex query -> GPT-4o ($2.50/M)
Very complex -> o3 ($10/M)

Strategy 2: Prompt Caching
Cache common prefixes -> 50% cost reduction
OpenAI caches prompts repeated within 5-10 minutes

Strategy 3: Batch API
Process non-urgent requests in batch
50% cost reduction, 24-hour turnaround

Strategy 4: Output Length Control
Set max_tokens appropriately
Shorter outputs = lower cost

1. Hallucination
   - Model can generate plausible but false information
   - Always verify critical facts
   - Use retrieval-augmented generation (RAG) for accuracy

2. Bias
   - Training data contains societal biases
   - Model may reflect stereotypes
   - Use diverse prompts and evaluate outputs

3. Knowledge Cutoff
   - Training data has a cutoff date
   - Cannot access real-time information (unless using tools)
   - Use web search for current events

4. Reasoning Errors
   - Complex multi-step reasoning can fail
   - Math calculations may be incorrect
   - Use o1/o3 for critical reasoning tasks

5. Privacy
   - Conversations may be used for training (opt out in settings)
   - Don't share sensitive personal information
   - Use API for data that shouldn't be stored

DO:
- Verify important information from multiple sources
- Disclose AI-generated content
- Respect intellectual property
- Use for augmentation, not replacement of human judgment
- Consider downstream impacts

DON'T:
- Use for deception or misinformation
- Share private/sensitive data
- Rely solely on AI for critical decisions
- Use to generate harmful content
- Ignore output quality (always review)