The 'dynamic languages are cheaper for LLMs' claim falls apart on real tasks

A popular and frequently-cited argument holds that dynamically-typed and terse languages cost LLM coding agents far fewer tokens than static languages like Rust, Go, or C++ — with claims of 2-3x savings, and array languages like J looking even cheaper still. Dan Luu traces those numbers back to their source and finds the underlying evals badly flawed. The headline comparisons rest on Rosetta Code snippets so trivial that a whole ‘problem’ fits in 70-109 tokens, and a second widely-shared benchmark contained a methodology bug where one agent symlinked a broken test path to its own executable, silently corrupting every later result. Performance on toy problems, he argues, simply doesn’t generalize.

To test this, Luu ran his own eval: implement a complete zstd decoder from the RFC alone, with no test suite provided and no internet access, using GPT-5.6 Sol at both medium and ultra reasoning effort. The clean dynamic-beats-static separation only appears at medium effort; crank the effort up and the results scramble, with several static languages landing among the best performers on both cost and correctness. The dramatic token ratios seen on trivial tasks evaporate once the work requires actual reasoning, surviving only where you’d expect poor results anyway — assembly, and obscure languages the AI labs have spent little or no RL training effort on.

The practical takeaway cuts against the viral advice. A weak-to-moderate positive correlation links language popularity to both cheaper and more-correct agent solutions, meaning mainstream languages are the safer bet for ordinary users. Exotic dense languages like J might pay off only for someone with the budget to fine-tune a model around them. The broader lesson is a recurring one in Luu’s benchmarking series: it is very easy to build an eval that appears to prove something it doesn’t, and small design differences between closely-related evals can flip the conclusion entirely.