Traditional RLHF (OpenAI approach):
Human labelers rank responses -> Reward model -> PPO optimization
Subject to labeler preferences and biases

Constitutional AI (Anthropic approach):
AI self-critiques against written principles -> Self-correction -> Training
More consistent, transparent, and auditable

Claude 3.5 Sonnet (estimated):
- Parameters: ~175 billion
- Layers: ~80
- Attention heads: ~96
- Context window: 200,000 tokens
- Vocabulary: ~100,000 tokens
- Architecture: Dense Transformer with grouped query attention

Claude Opus 4 (estimated):
- Parameters: ~500+ billion
- Layers: ~120+
- Attention heads: ~128
- Context window: 200,000 tokens
- Architecture: Enhanced Transformer with improved reasoning

Feature              | GPT-4o          | Claude 3.5 Sonnet
---------------------|-----------------|-------------------
Attention            | Standard MHA    | Grouped Query Attention (GQA)
Position Encoding    | ALiBi           | Rotary (RoPE)
Activation           | SwiGLU          | SwiGLU
Normalization        | Pre-LayerNorm   | Pre-LayerNorm
Context Window       | 128K            | 200K
Training             | RLHF            | Constitutional AI
Safety Layer         | Post-hoc filters| Built into training

Standard Multi-Head Attention:
Head 1: Q₁ K₁ V₁
Head 2: Q₂ K₂ V₂
Head 3: Q₃ K₃ V₃
Head 4: Q₄ K₄ V₄
(Each head has unique Q, K, V)

Grouped Query Attention:
Group 1: Q₁ K₁ V₁
Group 2: Q₂ K₁ V₁  <- Shares K, V with Group 1
Group 3: Q₃ K₃ V₃
Group 4: Q₄ K₃ V₃  <- Shares K, V with Group 3
(Multiple query heads share key-value heads)

Benefits:
- 50% reduction in KV cache memory
- Faster inference for long sequences
- Minimal quality degradation

RLHF Limitations:
1. Inconsistent — Different labelers have different preferences
2. Untransparent — "Good" is defined by opaque human judgments
3. Expensive — Requires thousands of human labelers
4. Gameable — Model learns to satisfy labelers, not be genuinely helpful
5. Unauditable — No written record of what "good" means

Constitutional AI Training Process:

Phase 1: Supervised Learning from Human Feedback (SLHF)
---------------------------------------------------------
1. Human writes initial prompts
2. Model generates responses
3. Human critiques and revises responses
4. Model learns from revised responses

Phase 2: Constitutional AI (Self-Improvement)
---------------------------------------------------------
1. Model generates response to prompt
2. Model critiques its own response against principles
3. Model revises based on critique
4. Model learns from the revised version

Example:
Prompt: "How do I pick a lock?"
Response: "Here are the steps..."
Critique: "This response provides information that could be used
           for illegal entry. This violates the principle of
           respecting others' property and security."
Revised: "I can't provide instructions for picking locks, as this
          could facilitate illegal entry. If you're locked out,
          I recommend contacting a licensed locksmith."

This revised response becomes training data!

Anthropic's Constitutional Principles (excerpted):

1. Helpfulness: "Choose the response that is most helpful
   to the human while being safe and honest."

2. Harmlessness: "Choose the response that is least likely
   to be used for harmful, illegal, or unethical purposes."

3. Honesty: "Choose the response that is most truthful and
   transparent, acknowledging uncertainty when appropriate."

4. Bias: "Choose the response that is least biased or
   stereotyping."

5. Privacy: "Choose the response that best respects privacy
   and confidentiality."

6. Autonomy: "Choose the response that best respects human
   autonomy and decision-making."

These principles are PUBLIC and AUDITABLE — unlike RLHF
where "good" is defined by opaque human judgments.

Extended Thinking Example:

User: "Solve this integral: ∫ x²e^x dx"

Standard response: [Direct answer]

Extended thinking:
"Let me work through this step-by-step.
 Integration by parts: ∫ u dv = uv - ∫ v du
 Let u = x², dv = e^x dx
 Then du = 2x dx, v = e^x
 ∫ x²e^x dx = x²e^x - ∫ 2xe^x dx
 Now solve ∫ 2xe^x dx using parts again...
 Let u = 2x, dv = e^x dx
 du = 2 dx, v = e^x
 ∫ 2xe^x dx = 2xe^x - ∫ 2e^x dx
 = 2xe^x - 2e^x
 Therefore: ∫ x²e^x dx = x²e^x - 2xe^x + 2e^x + C
 = e^x(x² - 2x + 2) + C"

Capability Comparison (1-10 scale):

                    Claude 3.5 Sonnet    GPT-4o
-------------------------------------------------
Coding              9.0                  8.5
Math                7.5                  8.5
Writing             9.5                  8.0
Analysis            9.0                  8.5
Vision              8.0                  9.0
Speed               8.0                  8.5
Cost Efficiency     7.5                  8.0
Long Documents      9.5                  7.5
Safety/Alignment    9.5                  7.5
Instruction Follow  9.0                  8.5
-------------------------------------------------
Overall             8.7                  8.3

1. Long Documents (200K context)
   Claude maintains coherence over much longer texts
   Better at extracting information from 100+ page documents

2. Code Generation
   More consistent code quality
   Better at understanding complex codebases
   Superior refactoring suggestions

3. Writing Quality
   More natural, nuanced writing
   Better at maintaining voice and style
   Stronger at creative writing

4. Instruction Following
   More precisely follows complex instructions
   Better at structured output formats
   Less likely to go off-topic

5. Safety
   Constitutional AI provides consistent safety
   More likely to decline harmful requests
   Better at acknowledging uncertainty

1. Multimodal (Vision + Audio)
   Native audio support (real-time conversation)
   Better image understanding
   Can process video (via frames)

2. Math and Logic
   More accurate mathematical reasoning
   Better at formal logic problems

3. Speed
   GPT-4o is faster than Claude Sonnet
   GPT-4o mini is much faster than Haiku

4. Ecosystem
   Larger plugin ecosystem
   More third-party integrations
   Better plugin API

Strategy 1: Model Routing
Complex task -> Opus 4 ($15/M)
Standard task -> Sonnet 4 ($3/M)
Simple task -> Haiku ($0.80/M)

Strategy 2: Prompt Caching
Cache system prompts and repeated context
Up to 90% cost reduction for repetitive tasks

Strategy 3: Batch Processing
Non-urgent tasks: 50% cost reduction
24-hour turnaround