A Browser Visualizer That Shows Which Past Tokens an LLM Actually Attends To

A developer built an interactive web tool that exposes the attention mechanism inside a transformer LLM, letting you hover over any generated token and watch the earlier tokens it drew from light up. To fit the model’s internal state into a single visible signal, the tool collapses a lot of detail into one number per past token — attention weight scaled by the value vector’s magnitude, aggregated across every head and layer — and maps that to opacity. Despite throwing away most of the underlying information, the result surfaces legible patterns, including cases where a generated phrase blends meaning from two separate source phrases.

The more interesting payoff is conceptual. The visualization offers an intuitive answer to why probabilistic next-token models are so reliable at reproducing verbatim strings like addresses, dates, and code: rather than reconstructing a sequence from limited internal state, the model can point back at the exact source tokens and copy from them, which keeps the error rate low. A 600M-parameter model in the demo reproduces an entire JavaScript function with only the intended edit, illustrating the copy-from-context behavior directly.

The build itself is a React app running Transformers.js, but standard inference wouldn’t expose the intermediate attention values needed for the display. Because Transformers.js executes a compiled ONNX graph in WebAssembly, only predefined outputs are reachable, so the author patched the ONNX file to surface the internal tensors and hosts the instrumented model on their own Hugging Face repo. Since the model is hundreds of megabytes, prompts are pre-generated so the visualizations load instantly, and the code is open on GitHub.