Probing Claude and GPT to Reverse-Engineer Their Training Timelines

A researcher shows that frontier models leak clues about how they were built simply by answering questions over their public APIs. The core method: build quizzes from Wikipedia daily-fact pages, give each model an 8-way multiple-choice test on what happened on specific dates, and watch where its accuracy collapses. That error-rate cliff approximates the model’s effective knowledge cutoff — and, by proxy, when its pre-training checkpoint was baked. Related techniques mentioned but covered elsewhere estimate parameter counts (“Incompressible Knowledge Probes”) and infer dataset mixtures from tokenizer behavior (“Data Mixture Inference”).

The estimates suggest Anthropic’s Opus 4.7 and later share a single pre-training run cutting off around late December 2025, while OpenAI’s GPT-5.6 family sits on a separate checkpoint finishing near late February 2026. One anomaly stands out: Opus 5 advertises cutoffs as late as May 2026 but demonstrates no real knowledge — including code package versions — past January 2026. Asking models what “today” is roughly tracks the fact-based estimates, and smaller distilled variants tend to report older dates, hinting at post-training on recency-skewed data.

The spicier finding comes from identity probing. When pushed to say what they are, models echo whatever appeared most in training — OpenAI models cycle through GPT-4/4o/4.1 and “ChatGPT,” Anthropic’s shift from Sonnet 3.5 to Sonnet 4.5 — strong evidence that labs train on their own users’ chat logs. Notably, Anthropic’s Sonnet 5 sometimes identifies as GPT-4, likely from ChatGPT conversations bleeding into the corpus, whether directly or through web contamination. In a follow-up test, Claude models reproduced OpenAI’s measured quirks 68% of the time when asked to imitate them, versus 8% the other way. The author stresses all of this is speculation, given how little ground truth labs publish.