TLDR IT · MISCELLANEOUS
CAN KAFKA SUPPORT ELASTIC PARTITIONING?
Summary from the TLDR IT newsletter
Kafka could support elastic partitioning, dynamically splitting and merging key ranges to scale ingestion and processing while balancing load. The design could preserve per-key ordering and Kafka's batch-oriented efficiency, but it would complicate consumer state locality, repartitioning, replay tracking, and recovery. Eliminating partitions entirely would sacrifice important batching and storage benefits.