Cantrill: AI extinction panic is fear sown by experts who should know better

Bryan Cantrill opens with a decades-old confession: as an arrogant first-year CS student, he and friends barged into a lab of humanities students and falsely announced that a computer virus was spreading, telling everyone to eject their floppy disks. The resulting panic — powered-off machines, lost term papers the week before finals, furious students — taught him that once fear takes root, it can’t be recalled, no matter how quickly you shout ‘prank.’ He uses the episode as a frame for what he sees happening now with AI.

The trigger is a claim by ex-Anthropic employee Jacob Coxon, endorsed by Anthropic Alignment Science lead Evan Hubinger, that there is a greater than 10% chance AI will ‘kill all humans’ within the next decade — not harm millions, but wipe out the species. Cantrill notes the supporting mechanisms (hacking critical infrastructure, engineered bioweapons) are asserted without elaboration by people who aren’t experts in those fields. What Coxon is actually demonstrating, he argues, is the contagion of fear: extinction dread spreads faster than evidence, and the sheer number of frightened technologists becomes mistaken for consensus, drowning out skeptics.

Cantrill’s rebuttal draws on his own domain — building computers. Engineering is not intelligence alone; it requires acting on and reasoning about the physical world, which today’s robots cannot do on anything like the timeline these fears assume. Any physical agency a digital system has is granted by humans operating systems built with human accountability and control. Invoking Sagan’s dictum that extraordinary claims demand extraordinary evidence, he argues the burden lies entirely with those raising the alarm. Because the public reasonably trusts domain experts, those experts have a duty to be circumspect — and most of all when sounding an alarm about human extinction.