RC RANDOM CHAOS

Karpathy retires the pelican test with a $10, two-hour Lord of the Rings render

· via Hacker News

Original source

Karpathy’s Pelican

Hacker News →

Andrej Karpathy argues that quick LLM benchmarks like “draw an SVG of a pelican on a bicycle” are running out of road, and proposes a heavier-weight alternative. He handed Opus 5 the opening paragraph of The Lord of the Rings, a one-million-token budget worth roughly $10, and asked it to produce a three.js rendering of the scene. The model worked autonomously for about two hours and generated 5,500 lines of code that procedurally animate the story—placing polygon assets in 3D space and orchestrating the whole thing. The result is rough around the edges, but the fact that it coheres at all is the point.

The broader thesis is about economics of effort rather than raw quality. Tasks that no human would rationally undertake because they are too tedious and bespoke become trivially cheap when a model with unlimited patience can grind through them for a few dollars. Karpathy sketches where this leads: hyper-custom, on-demand interactive worlds—an “ephemeral GTA” of any setting you name, where you could drop into Hobbiton as a spectator or a character.

He also flags a real limitation exposed by this kind of work. LLMs can’t natively perceive video or play through the games they build, so they can’t efficiently check their own visual output. Opus had to fall back on slowly capturing screenshots at various points to inspect results, and it stumbled repeatedly, producing much of the jank. Multimodal perception and interactive gameplay, in other words, remain a weak spot even as code-generation stamina keeps climbing.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.