PYTHON EXTENSIONS SHOULD BE LAZY

Summary from the TLDR Web Dev newsletter

Python's "everything is an object" philosophy slows down the parsing of large codebases with the ast module due to the overhead of converting C structs to Python objects. This conversion results in excessive memory allocations and garbage collection. To address the issue, this developer implemented a Rust extension that processed the ASTs in Rust, only returning the final results to Python, resulting in a 16x speedup and significantly reduced memory pressure. They argue that Python extensions should be designed to be lazy, meaning that they should only create Python objects when explicitly required, and instead keep data in a compact, optimized format within their own language.