FROM MILLIONS TO BILLIONS
Geocodio needed to scale its request logging from millions to billions of requests monthly for billing and customer support, but its deprecated MariaDB setup with TokuDB was struggling. Its initial attempt to replace MariaDB with ClickHouse directly resulted in "TOO_MANY_PARTS" errors due to small, frequent inserts, and a subsequent attempt using ClickHouse buffer tables led to "TOO_MANY_LINKS" errors. The solution involved introducing Kafka for durable storage and high throughput and Vector to efficiently batch and load data from Kafka into ClickHouse.