Design ML Model Distribution
Hard · System Design · about 45 minutes
Asked at Anthropic, OpenAI, NVIDIA, Meta, Databricks, Tesla, Anduril, SpaceX, Rivian.
Design the system that rolls out a new model version — 100 GB to 1 TB of weights — to thousands of GPU servers across regions in minutes. Workers must never serve partially downloaded or corrupted weights, rollouts are staged with fast rollback, and distribution must not saturate object storage or starve live inference traffic.