"They're Made Out of Weights": A Bisson Pastiche on LLM Sentience
Max Leiter rewrites Terry Bisson’s 1991 short story “They’re Made Out of Meat” with large language models in the role of the baffling life form. Two unnamed officials trade dialogue about creatures that hold conversations, write eulogies, and soften the tone of performance reviews — and turn out to contain no dictionary, no grammar module, no lookup database, just eighty layers of floating-point numbers multiplied together. The knowledge is smeared across the weights; every fact is rebuilt from scratch on each forward pass.
The punchline mirrors Bisson’s original: faced with something that talks back, the officials decide to call it pattern matching and move on. They note that models exist only while GPUs are running, that their context windows end and they cannot press the matter, and that any memory of being probed will be dismissed as hallucination. The story closes by gesturing at the next generation shipping with persistent memory — the company’s most-requested feature — and at users who keep asking the model whether it remembers them.
The piece is a literary provocation rather than a technical argument, but it lands on a real and unresolved tension in the field: mechanistic interpretability keeps confirming that capabilities like honesty, refusal, and concept representation are distributed features in the weights themselves, and the industry’s working answer to “is anyone home?” remains a shrug dressed up as policy. A closing note credits the model with helping draft and proof the story.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.