System Design Problems
Design a Chat System
A chat system enables real-time messaging between users. Services like WhatsApp, Slack, and Discord handle billions of messages daily with sub-second delivery, presence tracking, and end-to-end encryption.
- Real-time Delivery β Messages delivered in < 100ms between online users
- Persistence β Message history stored durably and searchable
- Group Support β 1-on-1 and group conversations with thousands of members
The core challenge is maintaining persistent WebSocket connections for millions of concurrent users while ensuring message ordering and delivery guarantees.
Requirements
Functional Requirements
- 1-on-1 messaging between users
- Group messaging (up to 1000 members)
- Online/offline presence indicators
- Read receipts and typing indicators
- Media sharing (images, files, voice messages)
- Message history and search
- Push notifications for offline users
Non-Functional Requirements
- Latency: Message delivery < 100ms for online users
- Ordering: Messages within a conversation are strictly ordered
- Durability: No messages lost; persisted to database
- Consistency: Eventual consistency across devices
- Scalability: 500M users, 50M concurrent connections
Back-of-the-Envelope Estimation
API Design
POST /api/v1/conversations
Request: { "type": "group", "members": ["u1", "u2"], "name": "Project Team" }
Response: { "conversation_id": "c_123" }
GET /api/v1/conversations/{id}/messages?before=msg_456&limit=50
Response: { "messages": [...], "has_more": true }
WebSocket: wss://chat.example.com/ws?token={jwt}
β Send: { "type": "message", "conversation_id": "c_123", "content": "Hello!" }
β Recv: { "type": "message", "from": "u_456", "content": "Hello!" }
High-Level Architecture
Detailed Design
WebSocket Connection Management
Maintaining persistent connections requires a stateful gateway:
Message Flow
- Sender's gateway receives message via WebSocket
- Gateway publishes to Kafka topic (partitioned by conversation_id)
- Chat service processes message (validation, persistence)
- Chat service looks up recipient's gateway
- If recipient is online, deliver via their gateway's WebSocket
- If recipient is offline, send push notification
Message Ordering
Ensure messages within a conversation are ordered:
Presence System
Track online/offline status using Redis:
// User comes online
SET presence:user_123 "online" EX 300 // 5-minute TTL
// Heartbeat (renew every 60 seconds)
EXPIRE presence:user_123 300
// Check presence
GET presence:user_123 // Returns "online" or nil
Read Receipts
Track which messages have been read:
// Store last read message ID per user per conversation
SET read:user_123:conv_456 msg_789
// When recipient reads messages, update
SETRANGE read:user_123:conv_456 msg_790
// Sender queries read status
GET read:user_123:conv_456 // Returns last read msg ID
Practice Exercises
-
Design: How would you implement end-to-end encryption for a chat system? What are the key management challenges?
-
Scale: If a group chat has 1000 members and one user sends a message, estimate the fan-out cost. How would you optimize for large groups?
-
Reliability: Design a system to ensure exactly-once message delivery. How do you handle duplicate messages from network retries?
-
Search: How would you implement full-text search across millions of chat messages? What indexing strategy would you use?
What to Learn Next
-> Networking Fundamentals TCP/IP, WebSockets, and connection management.
-> Message Queues Kafka, RabbitMQ, and event-driven messaging.
-> Design News Feed Fan-out strategies and real-time content delivery.
-> Design Notification System Multi-channel notification delivery and retry logic.
-> Databases Choosing Cassandra for time-series message data.
-> Caching Strategies Session management and presence tracking with Redis.