TRAINED ON TOKENS, CALIBRATED ON CONCEPTS: THE EMERGENCE OF SEMANTIC CALIBRATION IN LLMS

Summary from the TLDR AI newsletter

Semantic calibration appears to emerge as a byproduct of next-token prediction. Base models are remarkably well-calibrated when using a certain sampling-based notion of semantic calibration. They can meaningfully assess confidence in open-domain question-answering tasks despite not being explicitly trained to do so.