DICER

Summary from the TLDR Dev newsletter

Dicer is Databricks' auto-sharder, a foundational infrastructure system for building sharded services with low latency, high availability, and cost efficiency at scale. It addresses the inefficiencies of stateless models by colocating in-memory state with computation, so there are no issues like network latency and unnecessary data fetching.