DON'T LET THE LLM SPEAK, JUST PROBE IT

Summary from the TLDR AI newsletter

Answers often already exist in LLMs' hidden state before they generate a single token. It's possible to skip generation entirely by grabbing the hidden state at the last prompt token, feeding it to a tiny multilayer perceptron, and calibrating the output. This results in a frozen model that acts as any classifier that can be written in English.