LLM Fine-Tuning: LoRA, QLoRA, and Full Fine-Tuning Compared
Fine-tuning large language models has become more accessible with techniques like LoRA and QLoRA, but choosing the right approach depends on your budget, dataset size, and performance requirements. This guide compares all major fine-tuning approaches with practical implementation code.
The Fine-Tuning Spectrum
Full Fine-Tuning ←→ Parameter-Efficient Fine-Tuning (PEFT)
↓ ↓
Most expensive, Cheapest, fastest
best quality good quality
Full Fine-Tuning
Updates all model parameters. Requires the most resources but achieves the best results.
Requirements
- GPU: 4x A100 80GB (for 8B model)
- RAM: 128GB+
- Storage: 500GB+ (model checkpoints)
- Cost: ~$50-100/hour
- Time: 4-24 hours depending on dataset
LoRA (Low-Rank Adaptation)
Freezes the original model and trains small rank-decomposition matrices. Reduces trainable parameters by 90%+.
LoRA Hyperparameters
| Parameter | Range | Effect |
|---|---|---|
r (rank) | 4-64 | Higher = more capacity, more parameters |
lora_alpha | 8-64 | Scaling factor (usually 2x rank) |
lora_dropout | 0.05-0.2 | Regularization |
target_modules | varies | Which layers to adapt |
QLoRA (Quantized LoRA)
LoRA with 4-bit quantization. Fine-tune 70B models on a single 48GB GPU.
QLoRA vs LoRA Performance
| Metric | LoRA (16-bit) | QLoRA (4-bit) | Difference |
|---|---|---|---|
| Model quality | Baseline | -0.5-1% | Negligible |
| Memory usage | 32GB | 12GB | 62% less |
| Training speed | 1x | 0.7x | 30% slower |
| Inference speed | 1x | 0.9x | 10% slower |
RLHF (Reinforcement Learning from Human Feedback)
Fine-tune models to align with human preferences.
DPO (Direct Preference Optimization)
Simpler alternative to RLHF — no reward model needed.
RLHF vs DPO Comparison
| Aspect | RLHF | DPO |
|---|---|---|
| Complexity | High (3 models) | Low (2 models) |
| Training stability | Can be unstable | More stable |
| Reward hacking | Possible | Less likely |
| Performance | Slightly better | Very competitive |
| Compute cost | Higher | Lower |
Cost Comparison
| Method | GPU Required | Time (10K examples) | Cost |
|---|---|---|---|
| Full Fine-Tuning | 4x A100 80GB | 4-8 hours | $200-400 |
| LoRA (8B) | 1x A100 80GB | 2-4 hours | $50-100 |
| QLoRA (8B) | 1x A100 40GB | 3-6 hours | $30-60 |
| QLoRA (70B) | 1x A100 80GB | 8-16 hours | $100-200 |
| DPO | 2x A100 80GB | 4-8 hours | $100-200 |
| RLHF | 4x A100 80GB | 12-24 hours | $300-600 |
When to Use Each Method
Use Full Fine-Tuning when:
- You have a large dataset (100K+ examples)
- Maximum quality is critical
- Budget allows for significant compute
- You're fine-tuning a small model (<3B parameters)
Use LoRA when:
- You have a medium dataset (10K-100K examples)
- You need good quality with lower cost
- You want to preserve base model capabilities
- You need to fine-tune for multiple tasks
Use QLoRA when:
- You have limited GPU budget
- You're fine-tuning a large model (8B-70B)
- You want to experiment quickly
- Training speed is less critical
Use DPO/RLHF when:
- You want to align model with human preferences
- You have preference data (chosen/rejected pairs)
- Safety and alignment are important
Production Deployment
Serving LoRA Models
Conclusion
The choice of fine-tuning method depends on your specific constraints:
- QLoRA is the best starting point for most teams — low cost, good quality
- LoRA is ideal for production systems needing multiple adapters
- Full fine-tuning is for maximum quality when budget allows
- DPO is the simplest way to align with human preferences
- RLHF is for cutting-edge alignment when DPO isn't enough
The trend is toward parameter-efficient methods. With LoRA and QLoRA, you can fine-tune state-of-the-art models on a single GPU.