EXPLORING CLAUDE/GPT KNOWLEDGE CUTOFFS & PRE-TRAINING TIMELINES

Summary from the TLDR AI newsletter

It's possible to learn hidden facts about how frontier models were trained by probing them with carefully curated requests. The number of parameters models have can be estimated by scoring them on niche facts. Measuring how models break down tokens can reveal facts about the dataset mixtures used to train the model. Training timelines can be estimated by scoring models on date- or self-identification-related questions.