Recommendation System: Candidate Retrieval
How do you narrow down 10 million items to 1000 candidates in under 50ms? The art of fast retrieval at scale.
ML System Design
Production-grade machine learning system designs covering end-to-end architecture, scalability, and real-world engineering trade-offs. Learn how to build ML systems that serve millions of users.
Each design includes:
- Functional and non-functional requirements
- High-level architecture with diagrams
- Component deep-dives and implementation details
- Scaling and optimization strategies
- Monitoring, evaluation, and failure handling
- Trade-off analysis and decision frameworks
Browse by Domain
Recommendation Systems:
Classification Systems:
Data Infrastructure:
Experimentation & Metrics:
Model Serving:
Model Evaluation:
Feature Engineering:
Model Deployment:
Model Training:
Data Augmentation:
MLOps & Experiment Tracking:
AutoML & Model Design:
Infrastructure:
Model Optimization:
Unsupervised Learning:
Real-Time Systems:
Search & Ranking:
- Coming soon…
Computer Vision:
- Coming soon…
Natural Language Processing:
- Coming soon…
Real-Time Systems:
- Coming soon…
Feature Engineering & Stores:
- Coming soon…
Model Serving & Deployment:
- Coming soon…
System Design Index
Below you’ll find all ML system design problems in chronological order:
Content created with the assistance of large language models and reviewed for technical accuracy.
Classification Pipeline Design
From raw data to production predictions: building a classification pipeline that handles millions of requests with 99.9% uptime.
Data Preprocessing Pipeline Design
How to build production-grade pipelines that clean, transform, and validate billions of data points before training.
A/B Testing Systems for ML
How to design experimentation platforms that enable rapid iteration while maintaining statistical rigor at scale.
Batch vs Real-Time Inference
How to choose between batch and real-time inference, the architectural decision that shapes your entire ML serving infrastructure.
Model Evaluation Metrics
How to measure if your ML model is actually good, choosing the right metrics is as important as building the model itself.
Feature Engineering at Scale
Feature engineering makes or breaks ML models, learn how to build scalable, production-ready feature pipelines that power real-world systems.
Model Serving Architecture
Design production-grade model serving systems that deliver predictions at scale with low latency and high reliability.
Online Learning Systems
Design systems that learn continuously from streaming data, adapting to changing patterns without full retraining.
Caching Strategies for ML Systems
Design efficient caching layers for ML systems to reduce latency, save compute costs, and improve user experience at scale.
Content Delivery Networks (CDN)
Design a global CDN for ML systems: Edge caching reduces latency from 500ms to 50ms. Critical for real-time predictions worldwide.
Distributed ML Systems
Design distributed ML systems that scale to billions of predictions: Master replication, sharding, consensus, and fault tolerance for production ML.
Resource Allocation for ML
Build production ML infrastructure that dynamically allocates resources using greedy optimization to maximize throughput and minimize costs.
Model Ensembling
Build production ensemble systems that combine multiple models using backtracking strategies to explore optimal combinations.
Clustering Systems
Design production clustering systems that group similar items using hash-based and distance-based approaches for recommendations, search, and analytics.
Event Stream Processing
Build production event stream processing systems that handle millions of events per second using windowing and temporal aggregation—applying the same interva...
Distributed Training Architecture
Design distributed training architectures that can efficiently process massive sequential datasets and train billion-parameter models across thousands of GPUs.
Data Augmentation Pipeline
Design a robust data augmentation pipeline that applies rich transformations to large-scale datasets without becoming the training bottleneck.
Experiment Tracking Systems
Design robust experiment tracking systems that enable systematic exploration, reproducibility, and collaboration across large ML teams.
Online Learning Systems
Design online learning systems that adapt models in real-time using greedy updates—the same adaptive decision-making pattern from Jump Game applied to stream...
Neural Architecture Search
Design neural architecture search systems that automatically discover optimal model architectures using dynamic programming and path optimization—the same pr...
Cost Optimization for ML
A comprehensive guide to FinOps for Machine Learning: reducing TCO without compromising accuracy or latency.
Beam Search Decoding
The industry-standard algorithm for converting probabilistic model outputs into coherent text sequences.
Tokenization Systems
The critical preprocessing step that defines the vocabulary and capabilities of Large Language Models.