The LLMentalist Effect: Why Chatbots Feel Smart Is a Cold-Reading Con

Baldur Bjarnason argues that the widespread conviction that chat-based language models are intelligent is a perceptual illusion, not a property of the models themselves. LLMs are mathematical models of language that return statistically plausible continuations of text; nothing in their architecture reasons or thinks, a point vendors and researchers themselves repeatedly stress. Faced with two explanations—that the industry accidentally birthed a novel form of mind, or that the intelligence lives in the user’s head—he lands firmly on the latter, and names the mechanism.

That mechanism, he contends, is cold reading: the same trick mentalists and fake psychics use to seem uncannily perceptive. By emitting confident, specific-sounding statements that are actually statistically generic (the Forer effect), a chatbot creates the impression that it is engaging personally with you and your work. The piece maps the classic psychic’s con step by step—a self-selecting, credulous audience, a primed setting, gradual narrowing to a receptive ‘mark,’ and a subjective-validation loop in which plausible guesses phrased with authority get read as insight—onto how users come to believe an LLM understands them. Subjective validation, the human tendency to rate any statement we personally relate to as accurate, does the rest.

The stakes, for Bjarnason, go beyond ordinary tech-bubble hype: the awe-and-dread tone of AI true believers echoes the language of scam victims, and he now sees many proposed LLM use cases as bordering on pseudoscience. Framing anthropomorphic claims about model ‘reasoning’ as an accidental automation of a centuries-old grift, the essay is a caution against mistaking fluent, tailored-sounding output for genuine cognition.