RC RANDOM CHAOS

Bigger isn't better: open GLM-5.2 hallucinates ~3x less than giant GPT-5.5

· via Hacker News

Original source

GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

Hacker News →

The piece argues that the parameter-scaling era is hitting diminishing returns. Its headline evidence: Z.ai’s MIT-licensed GLM-5.2 (753B total, ~40B active) lands within 4 points of GPT-5.5 and 9 of Fable 5 on the Artificial Analysis Intelligence Index, despite proprietary leaders being an estimated 1–2T parameters. If a much smaller open-weight model can sit that close, the author contends, raw ‘intelligence’ has effectively plateaued. The author also opens with a striking — and unverified — anecdote that Claude Fable 5 was restricted by the US government days after release over a single jailbreak, framing it as a sign that ever-larger frontier models carry outsized risk.

The more interesting claim concerns calibration. On the AA-Omniscience hallucination benchmark, the biggest models fare worst: DeepSeek V4 Pro confidently invents answers 94% of the time it doesn’t know, GPT-5.5 hits 86%, while GLM-5.2 (28%) and Opus 4.8 (36%) are far better at admitting uncertainty. The author’s reasoning is that models saturated with factual training data learn to always produce an answer rather than say ‘I don’t know.’ A hand-rolled test reinforces the point: given a Python task with a built-in architectural impossibility (single-threaded multiplexed I/O that never yields), DeepSeek burned ~10x the reasoning tokens and three-plus minutes to produce a polished but wrong solution, while GLM-5.2 flagged the contradiction in about 12 seconds.

The takeaway is a proposed ‘trilemma’ for modern LLMs — raw capability, hallucination/uncertainty calibration, and compute efficiency — that scaling alone can’t resolve and may actively worsen. The author urges both labs and users to stop selecting models by size or benchmark score alone. Worth noting: the specific figures, model names, and the government-ban anecdote are presented without primary sourcing and read as a mix of real benchmarking and speculative near-future framing, so the directional argument is stronger than any individual number.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.