RAPIDLY SCALING ONLINE STORAGE TO SERVE OVER 1 BILLION CHATGPT USERS

Summary from the TLDR Data newsletter

OpenAI's storage platform, Habitat, serves over 500 petabytes to over 1 billion users. Its Azure Cosmos DB setup went from one shared container to one per feature, then back to shared after teams forgot to pre-scale for launches. With hot key throttling, request priorities, and a new serverless mode, it has had no noisy-neighbour incidents since.