OpenAI details storage changes that cut median read latency twelvefold

OpenAI has detailed changes to Habitat, the storage platform supporting ChatGPT and its other products. In an October 23 engineering update, the company said median global read latency fell from approximately 120 milliseconds to 10 milliseconds.

The improvements combine nearby copies of critical data, regional caches and a rewrite of service code in Rust. Habitat uses Azure Cosmos DB, with database changes triggering cache updates across regions within seconds. OpenAI says the platform serves more than 500 petabytes across 40 geographic regions.

Reliability also required controls to prevent heavy workloads from disrupting unrelated requests. OpenAI described cases where demand from smaller features affected essential functions such as account registration and login. Its revised approach combines shared infrastructure with workload isolation and capacity management.

The slowest requests still face delays when uncached information must travel between distant regions. OpenAI plans to place more data near its users as capacity and storage efficiency improve.