Distillation makes fast followers, never frontier leaders

Garry Tan, the CEO of Y Combinator, told CNBC this week that US regulators should do nothing about Chinese labs distilling American frontier models. His follow-up to TechCrunch went further. He wants smaller American open-weight labs to run the same technique on American frontier labs, building what he called an “American distillation regime.”

Distillation here means extensively prompting another model to learn how it works and reasons. Labs do it routinely and legitimately to train new models. The mechanism is not what anyone is fighting about. Tan is responding to Anthropic’s second report, which alleges that Chinese labs run “illicit distillation attacks”: hiding their identities to distill without permission, and using fraud and stolen credentials for the access. Dario Amodei had already asked US regulators to crack down. Tan, running Silicon Valley’s prestigious and prolific startup accelerator, lands on the other side.

He is not arguing for stolen credentials. He wants American labs to distill through the front door, paying for API access and prompting as any customer would. His case has two parts. He thinks it is overreach for a lab to dictate what a paying customer does with the output of an API call. And he points out that the proprietary labs did not ask permission either when they ingested copyrighted material wholesale to train the models in the first place. “Access to intelligence that was trained on broad public access data should itself also be more a form of a public good than something locked away behind restrictive terms of service,” he told TechCrunch.

Underneath the politics, the technique has a built-in limit. A distilled model learns from a teacher’s outputs, so the teacher defines the ceiling of what is being copied. An open-weight lab whose strategy is distilling the current frontier is chasing a target someone else set. It can ship a cheaper, open model that lands close to the frontier it copied, but it cannot get ahead of the lab it is distilling by distilling it. The student approaches the teacher; it does not pass it.

Tan’s own framing concedes the split. Frontier labs are “at the frontier and driving it forward,” and he wants that to stay fundable and remain a good business. Open-weight models “give people freedom and access.” Those are two different jobs: one sets the pace, the other distributes what has already been set, widely and cheaply.

His actual worry is a single monolithic company with the best capital and the best researchers running away with the field. A distillation regime answers that by keeping cheap, open copies within reach of the frontier, so no one lab holds a runaway monopoly on capability. That is a genuine argument for distribution, not an argument that the followers will ever lead.

For anyone building on open weights, two things follow. The capability you inherit from a distilled model tracks the teacher’s, so your ceiling is set by a lab you do not control. And your legal exposure depends on how the access was obtained, which is the line Anthropic’s report is built around: paying customers prompting through the front door sit in a different position from anyone using fraud and stolen credentials. Tan wants that front door kept open and well-lit, for Americans at least.