EXTRACTING CONCEPTS FROM GPT-4

Summary from the TLDR AI newsletter

The team at OpenAI has discovered 16 million interpretable features in GPT-4 including price increases, algebraic rings, and who/what correspondence. This is a great step forward for SAE interpretability at scale. They shared the code in a companion GitHub repository.