HOW SLACK BUILT A DISTRIBUTED CRON EXECUTION SYSTEM FOR SCALE

Summary from the TLDR Web Dev newsletter

Slack initially relied on cron jobs for its notification system but found it hard to scale. To address this, its team developed a new cron execution service using Go and Kubernetes for scalability and a leader-follower architecture for simplified synchronization and rapid failover. This service offloads resource-intensive tasks to a job queue, using Kafka for durability and Redis for efficient dispatch to worker nodes, and tracks job execution with a Vitess table for deduplication and monitoring.