A 125M transformer that autocompletes piano in real time on an iPhone

A developer spent roughly a year building RollTab, a free iOS/iPadOS app that acts like Copilot for a MIDI piano: you play a short phrase and a small on-device transformer continues it. The final 125M-parameter, decoder-only model (RMSNorm, rotary embeddings, SwiGLU) runs at about 108 notes per second on an iPhone 15 — far faster than any live player needs. The author reports that the biggest gains came not from architecture tweaks but from three things: the MIDI representation, aggressive data cleaning, and DPO post-training.

The core engineering insight is the tokenization scheme. Naive note-on/note-off encodings caused the model to drift and leave notes hanging, while fully factored representations were accurate but slow, spending four autoregressive steps per note. The chosen format packs each note into a single token — NOTE(pitch, delta_onset, duration, velocity) — where silence is encoded as the delta to the next onset and chords share a delta of zero. Five categorical fields are summed into one embedding and decoded through a lightweight nested decoder with separate output heads, so the expensive transformer backbone advances the music one complete note per pass rather than one attribute at a time. Sustain-pedal behavior is folded into note durations during preprocessing to keep the modeling target simple.

Data quality proved decisive. The training set was a few hundred thousand public-domain, mostly classical MIDI files — around 300 million note events — heavily filtered, deduplicated by transposition- and tempo-invariant fingerprints, and reduced to piano-focused material. Scaling the corpus roughly 5x actually hurt performance, reinforcing that curation beat raw volume. Training used summed cross-entropy across the five heads plus augmentations (transposition, tempo scaling, jitter, dropped notes) to tolerate sloppy live input, since cross-entropy alone is a poor proxy for how a full continuation actually sounds.