CAN SAES CAPTURE NEURAL GEOMETRY?

Summary from the TLDR AI newsletter

Sparse autoencoders (SAEs) can help explore neural geometry by representing curved manifolds using several distinct methods, including shattering, compact capture, and dilution. The study reveals that each SAE feature only represents part of the whole manifold, necessitating the clustering of features to understand the overall structure. An unsupervised pipeline leveraging these findings could enhance our understanding of neural networks by reconstructing the internal geometric structures on their terms.