Design ChatGPT Playground
Medium · System Design · about 45 minutes
Asked at OpenAI, Anthropic, Databricks, Microsoft.
Design a developer playground for LLMs like the OpenAI Playground: developers write a system prompt and messages, pick a model and parameters, stream responses, compare models side by side, and save versioned prompts to share. Assume ~1M developers and ~100k concurrent streaming generations at peak; model inference is provided by an internal gateway.