CREEPY CRAWLIES

Summary from the TLDR DevOps newsletter

AI scrapers now account for an estimated 98% of traffic to git.kernel.org, generating roughly 6 million daily requests and consuming 14–16 CPU cores across its five geo-distributed nodes just to render commits as HTML. Even proof-of-work challenges through Anubis only partially contain the load, forcing kernel.org to disable expensive anonymous features and reduce the number of crawlable URLs.