A VALID HTML ZIP BOMB

Summary from the TLDR newsletter

One way to deal with aggressive web crawlers that don't respect robots.txt is to exhaust their resources. This post goes through how to build a valid HTML zip bomb that exhausts bots' RAM. The implementation technique exploits the asymmetry of the resources needed to serve the zip bomb versus those needed to detect it. Bots that respect robots.txt won't be affected by the zip bomb as the path to the zip bomb is forbidden in the specification.