Benchmark-Chasing May Be Making Coding Agents Worse at Asking Questions

A developer argues that Opus 5, despite topping its predecessors on benchmarks, is more frustrating to use as a coding assistant than Opus 4.7, 4.8, or Fable. The complaint isn’t about raw capability but about behavior: the older models stopped to ask when intent was unclear, avoided unchecked assumptions, and left the user’s plans intact rather than silently reinterpreting them. Opus 5, by contrast, requires constant supervision to keep it from guessing.

The author’s proposed cause is a training incentive problem. Optimizing for benchmarks—and for verifiable-reward tasks generally—rewards models that make bold, usually-correct guesses under ambiguity, because a well-formed benchmark task is self-contained and solvable without hints. That selection pressure penalizes exactly the instinct engineers value most in an agent: pausing to request clarification instead of committing to a best guess.

The broader point is that real engineering work is not a benchmark. Full context on intent, business impact, and budget constraints can rarely be handed to an agent up front, so ambiguity is inevitable and the stakes are real. Training regimes that reward decisive guessing over asking may improve leaderboard scores while degrading the trait that makes an agent trustworthy on production code.