System Design Problems
Design Google Search
Google processes 8.5B+ searches per day across 100M+ web pages. This design covers web crawling, inverted index construction, and the PageRank ranking algorithm.
- Scale â 8.5B searches/day, 100B+ pages indexed
- Index â Petabytes of inverted index data
- Latency â Search results in < 200ms
Google Search is the ultimate big data problem: crawling the entire web, building a searchable index, and ranking results in real-time.
Requirements Clarification
Functional Requirements
- Search web pages by keywords
- Return ranked results with snippets
- Support boolean operators (AND, OR, NOT)
- Autocomplete and spell correction
- Search images, videos, news
- Personalized results based on history
Non-Functional Requirements
- Availability: 99.99% uptime
- Latency: Results < 200ms
- Freshness: New pages indexed within hours
- Scale: 8.5B searches/day, 100B+ pages
Back-of-the-Envelope Estimation
High-Level Architecture
Web Crawler
Inverted Index
PageRank Algorithm
Query Processing Pipeline
Data Model
Practice Exercises
- Crawler Design: How would you prioritize crawling 1B new pages daily while respecting politeness limits?
- Index Design: Design a distributed inverted index that supports 100K QPS queries.
- Ranking: How would you combine PageRank with user-specific signals for personalized search?
- Freshness: Design a system that indexes breaking news within minutes of publication.
What to Learn Next
-> Design Leaderboard Sorted sets and real-time ranking.
-> Design Dropbox File storage and sync systems.
-> Design Amazon E-commerce search and catalog.
-> Design Twitter Trending topics and search.
-> Back Pressure Managing load in crawling systems.
-> Circuit Breaker Resilient web crawling.