Data-Oriented Design: Lay Out Data for the Cache, Not the Object Model

This DICE slide deck by Daniel Collin argues that game engine programmers should design around how data is read and written rather than around object hierarchies. The core motivation is memory latency: a main-memory fetch costs roughly 600 cycles on a 3.2 GHz CPU, while L1 cache and registers cost only 1–2 cycles, so most real code is bound by memory access, not computation. Classic object-oriented layout works against this. A typical Bot class scatters position, modifier, and aim fields across a fat object, so a single updateAim() call triggers instruction-cache and data-cache misses and drags in bytes it never uses. Run that method over four bots and the cache misses compound to about 7,680 cycles.

The data-oriented alternative is to design ‘back to front’ — start from the output you need, then pull in only the minimal input required to produce it. Rewriting the code as a loop over parallel linear arrays (updateAims over an array of positions and modifiers, writing into a contiguous output array) leaves the actual arithmetic unchanged but transforms the access pattern. After the first iteration warms the cache, subsequent elements are cheap, dropping the four-bot cost to roughly 1,980 cycles. The underlying shift is from array-of-structs to struct-of-arrays, which packs the fields actually being touched into the same 128-byte cache lines instead of wasting them on unused data.

Collin’s broader point is that this is a mindset, not a micro-optimization: not everything needs to be an object, and because games control their own data, runtime (‘native’) layouts can be pre-formatted independently of source formats — e.g. converting a linked list of area triggers into a flat array, or replacing a tree-based culling system with brute-force iteration over linear arrays that ran 3x faster in one-fifth the code. The payoff, he argues, is better performance, simpler code, and data that is far easier to multithread or offload to co-processors like the PS3’s SPUs, since you already know exactly how it is accessed.