Architecture
Design Outbox Pattern
The outbox pattern ensures reliable event publishing by writing events to a database outbox table in the same transaction as business data. A separate process polls the outbox and publishes to the message broker.
- Problem â Dual writes (DB + MQ) are not atomic
- Solution â Transactional outbox with CDC polling
- Guarantee â At-least-once delivery with idempotent consumers
The outbox pattern solves the fundamental problem: you can't write to a database and a message broker atomically.
The Dual Write Problem
Outbox Pattern Solution
Outbox Table Schema
Polling vs CDC
| Approach | Latency | Resource Usage | Complexity |
|---|---|---|---|
| Polling | 1-5 seconds | High (queries) | Low |
| CDC | < 1 second | Low (log tailing) | Medium |
| CDC + Kafka | < 100ms | Low | High |
Exactly-Once Delivery
Practice Exercises
- Design: Implement an outbox pattern for an order service with Debezium CDC.
- Migration: How would you migrate from polling to CDC without downtime?
- Ordering: How do you ensure events are published in order for a single aggregate?
- Scale: Design an outbox system that handles 100K events per second.
What to Learn Next
-> Saga Pattern Distributed transactions.
-> Idempotency Exactly-once semantics.
-> Back Pressure Event flow control.
-> Design WhatsApp Message delivery guarantees.
-> Design Instagram Event-driven feed generation.
-> Strangler Fig Migrating to event-driven architecture.