LLMs Collapse the Cost of Performance Engineering

Dan Luu argues that AI coding agents have turned performance optimization from specialized, expensive work into something almost anyone can commission in minutes. Techniques that once demanded rare expertise — JIT compilers, custom indexers, hand-tuned multithreading — were rationed not because they lacked value but because writing the code was prohibitively hard. With agents, that barrier largely disappears, opening the door to software fitted to a specific workload rather than a general class of them.

He walks through concrete experiments to make the case. Building on FRE, a regex engine an agent produced by looping for a month against the rebar benchmark suite, he had an agent bolt an ahead-of-time native compiler onto ripgrep: it compiles in a background thread and cuts over once ready, trading slightly slower short queries for faster long ones. On representative holdout queries the AOT path yields about a 7% speedup (2–4x on some simple long queries) — modest, but the point is that it took a few sentences of prompting rather than days of expert engineering. He notes the smarter move for repeated searches would be a full-disk index, drawing on his work on Bing’s BitFunnel, and suggests a fast whole-machine indexer is now a weekend-scale project.

A sharper example: with no game-AI background, Luu built what he says is the world’s strongest AI for the board game Azul, winning largely on optimization — multithreaded search, both native and WASM builds, and separate minimax/MCTS architectures — plus agent-generated debugging scaffolding like deterministic replay from logs. The broader thesis is that as inference gets cheaper and faster, ambitious, workload-specific optimization becomes routine, with its own set of new risks and opportunities.