DEEPDIVE: GETTING STARTED WITH MECHANISTIC INTERPRETABILITY

Summary from the TLDR AI newsletter

Neel Nanda is one of the pioneers of Mechanistic Interpretability (MI). This is his guide to getting into the field. It features 200 concrete open questions. MI is the study of the quantitative values of language models (e.g., actually looking at neurons). It is accessible as it doesn’t require significant computation, and is extremely promising as a research direction, although there has been minimal success to date.