LARGE LANGUAGE MODELS USE A SURPRISINGLY SIMPLE MECHANISM TO RETRIEVE SOME STORED KNOWLEDGE

Summary from the TLDR Web Dev newsletter

LLMs often use simple linear functions to retrieve stored facts. Researchers developed a technique to estimate these functions, allowing them to probe what the model knows and where that knowledge is stored, potentially leading to ways to correct falsehoods within the model.