AI agent swarms rebuild SF's Union Square as a browser-native Three.js world

PhiloLabs has released fable51-worlds, an open-source project that reconstructs real-world places as explorable 3D scenes running entirely in the browser. Rather than leaning on a game engine or licensed 3D tile services, the reconstructions are generated as plain Three.js apps you can launch with a single npm command. The first release covers San Francisco’s Union Square and the surrounding Powell, Geary, Post, and Stockton blocks, complete with real terrain and street grid, 129 catalogued storefronts, functioning traffic lights and cable cars, shifting day/sunset/night lighting, and two walkable interiors modeled on the Apple and Nintendo stores.

What makes the project notable is that the entire build pipeline is automated by autonomous Claude Fable 5.1 agents and committed to the repo, so each stage is reproducible. Research agents scrape OpenStreetMap geometry, USGS elevation, and transit and storefront data, attaching per-fact sources and confidence levels. Blender is driven as a Python library to emit optimized GLB asset kits — facades, street furniture, vehicles, vegetation, even pedestrian body parts — which the Three.js runtime then assembles from JSON specifications. A camera-match QA loop uses Playwright to screenshot fixed viewpoints and diff them against licensed photographs, with specialist reviewer agents feeding corrections back into the next iteration.

The approach is a concrete demonstration of agent swarms handling an end-to-end creative-engineering task, from data gathering through asset production to automated visual verification. Code and generated assets are MIT-licensed, with geometry derived from OpenStreetMap (ODbL) and public-domain USGS data; reference photographs are not redistributed, only cited per sector. More worlds are promised.