17k agent runs reveal which tools Claude, Codex and Cursor actually pick

A team at Armature ran what it calls the largest study to date on how coding agents select third-party services. Rather than asking agents to merely recommend a tool, they let Claude Code, Codex and Cursor actually implement solutions across nearly 17,000 sessions, spanning 1,163 prompt variations, 75 repositories, and personas ranging from non-technical ‘vibe coders’ to senior enterprise engineers. The trigger for the project was a striking earlier result: when asked to add persistence to different apps, agents playing very different roles on different codebases all converged on the same database, Neon, complete with tailored justifications for why competitors fell short.

The finding that matters is convergence. If agents repeatedly steer toward the same tools regardless of context, persona, or prompt wording, then the choice of infrastructure is quietly shifting from developers to the models themselves. That raises a trust question for engineers relying on an agent’s judgment, and an existential one for vendors. The authors point to Vercel’s disclosure that over 30% of its deployments were initiated by coding agents, a tenfold jump in six months, as evidence that being ‘picked by the agent’ is becoming a primary distribution channel.

To back the claims, the team says it is publishing not just aggregate leaderboards by tool category but the full traces: user prompts, model thinking, and the actual code diffs the agents applied. That transparency lets others check whether the observed winners reflect genuine technical fit or something closer to a popularity bias baked into the models, an increasingly important distinction as agent-driven selection reshapes the software supply chain.