WEIGHT-SPARSE TRANSFORMERS HAVE INTERPRETABLE CIRCUITS

Summary from the TLDR AI newsletter

OpenAI trained language models with most weights forced to zero, making it easier to identify which neurons link together into a “circuit” representing a certain concept. The approach successfully isolated circuits for tasks like Python quote-matching that are both necessary (removing them breaks the behavior) and sufficient (they work in isolation).