PERSONA VECTORS: MONITORING AND CONTROLLING CHARACTER TRAITS IN LANGUAGE MODELS

Summary from the TLDR AI newsletter

Researchers extracted "persona vectors" by comparing neural network activity when models exhibit specific traits versus when they don't, revealing that personality changes follow predictable mathematical patterns. This discovery enables a counterintuitive "vaccination" approach that purposefully steers models toward undesirable traits during training to compensate for their tendency to “overfit” to negative traits.