Daily Edition · Page 14

THE DAILY INDEX

Technology, AI, software, security, crypto and business

NEWS & TRENDS

TLDR DevOps

THE LIFECYCLE OF A SHARDED POSTGRES QUERY

PlanetScale's Neki router makes a sharded Postgres deployment appear as a single database by handling authentication, protocol parsing, shard-aware planning, connection pooling, and distributed execution across many Postgres nodes. Query performance depends heavily on shard-key choice: co-locating related rows can turn an expensive cross-shard join that requires scatter-gather queries and router-side hash joins into local joins executed independently on each shard.

RAPIDLY SCALING ONLINE STORAGE TO SERVE OVER 1 BILLION CHATGPT USERS

OpenAI's Habitat storage platform now handles over 70 million requests per second and serves more than 500 petabytes of data. The team recently rewrote the service in Rust, which is 6x more CPU efficient and 15x more memory efficient than the previous Python version. Habitat first launched as a Python library at DevDay 2023 to support GPTs and evolved into a complex distributed system supporting over 1 billion weekly users across nearly 40 regions.

TLDR IndexPage 14