Fable 5 dominates GPT-5.6 Sol on NP-hard task; /goal wins runs but hurts averages

A researcher pitted Claude Fable 5 against GPT-5.6 Sol on KIRO, an unpublished fiber-network optimization problem with an astronomically large search space (Paris alone yields a lower bound around 10^1223 valid configurations). Fable 5 was the standout: it posted the best clean score (31,934) and, more tellingly, near-total consistency—its plain-mode results stayed within a 319-point band while Sol’s swung across nearly 2,000 points. On means, Fable beat Sol by roughly 1,875–1,984 points depending on mode. The author, who once spent a week hand-writing C++ for this problem, frames Fable’s showing as raw problem-solving strength rather than clever scaffolding.

The more interesting finding concerns each tool’s /goal persistence feature. It won four of six matched trials, yet made both models’ average scores worse—a small typical gain paired with rare but severe regressions. /goal isn’t a generic ‘try harder’ toggle; it reshapes the control loop and search path, which helps when it sustains a good solver strategy and backfires when it lets a bad choice mature. The two implementations also diverge sharply: Claude Code runs /goal as a Stop hook where a separate small evaluator (Haiku) judges only the transcript and can’t inspect files, while Codex persists the goal as thread state and hands the working model tools to declare its own completion—independent-but-blind versus informed-but-grading-its-own-work.

The author is careful about scope: this is a single NP-hard task, only Fable and Sol got three clean matched pairs, and the containers exposed eight CPUs despite metadata claiming one, which favored Fable’s parallel solver portfolios. The takeaway worth generalizing isn’t whether /goal helps, but that a persistence feature can win most individual trials while degrading observed average performance. On hard optimization, the quality of what an agent keeps doing matters more than simply doing more of it.