ANALYZE FEATURE FLOW TO ENHANCE INTERPRETATION AND STEERING IN LANGUAGE MODELS

Summary from the TLDR AI newsletter

This paper introduces a novel approach to tracking how features discovered by sparse autoencoders evolve across consecutive layers of large language models using a data-free cosine similarity technique that maps feature persistence, transformation, and emergence. The paper demonstrates how the resulting cross-layer feature maps enable direct behavioral control of the models through feature manipulation while providing mechanistic insights into model computations through granular flow graphs.