Interactive Explainer Cracks Open the Transformer, GPT-2 Live in Your Browser

Georgia Tech’s Polo Club released Transformer Explainer, an interactive visualization that runs a live GPT-2 (small) model in the browser to demystify the architecture behind modern generative AI. The 124-million-parameter model is old and modest by today’s standards, but it shares the same structural DNA as GPT, Llama, and Gemini, making it a practical teaching vehicle for the mechanics everyone now depends on.

The walkthrough traces the full next-token prediction pipeline. Input text is tokenized against GPT-2’s 50,257-token vocabulary, mapped to 768-dimensional embedding vectors, and combined with learned positional encodings. Those embeddings then flow through 12 stacked Transformer blocks, each pairing multi-head self-attention—where Query, Key, and Value projections let tokens weigh their relevance to one another—with a per-token MLP that refines each representation. A final linear-plus-softmax layer converts the result into probabilities over the next token.

The piece is clearest on the details practitioners often gloss over: the web-search analogy for Q/K/V, the split into 12 attention heads that each capture different syntactic and semantic patterns, and the causal mask that zeroes out future tokens so the model can’t peek ahead during training. For engineers who use these models daily but treat attention as a black box, it’s a concrete, hands-on reference rather than another abstract diagram.