ASTRA DYNAMIC CHUNKS: HOW WE SAVED BY REDESIGNING A KEY PART OF ASTRA

Summary from the TLDR Web Dev newsletter

Slack's log search engine, Astra, initially used fixed-size data chunks, leading to wasted disk space and increased infrastructure costs due to underutilized chunks. To address this, the Slack team redesigned Astra to use dynamic chunks. This involved implementing a first-fit bin packing algorithm to optimize chunk assignment across cache nodes, using Zookeeper for persistent data tracking.