Branchless Rust: Killing One If Statement Makes a Filter 4x Faster

A simple Rust filter that keeps values above a threshold turns out to be slowest not when it copies the most data, but when its keep/skip decision is hardest to predict. Benchmarking one million random floats, Serhii Potapov found the 50%-selectivity case ran ~2.6x slower than the 99% case despite moving far less data. The cause isn’t allocation—preallocating the output vector shaved only 2%—but the CPU’s branch predictor. Modern pipelined cores speculate which way an if will go; when the branch depends on unpredictable data, roughly half a million mispredictions each cost 15–20 cycles in pipeline flushes, adding milliseconds of pure penalty. Sorting the input first (making the branch predictable) sped the same code up 4.5x, confirming the diagnosis.

The fix is branchless programming: instead of conditionally writing each element, always write it to out[n] and advance the cursor by (x > threshold) as usize—0 or 1. Rejected values simply get overwritten next iteration, and a final truncate(n) trims the tail. This converts a control dependency into a data dependency; the comparison compiles to a seta instruction that produces a number rather than a fork, so there’s nothing to mispredict. The worst case gets nearly 4x faster and, notably, its runtime becomes flat—independent of the data distribution.

The technique is a trade, not free speed. At 1% selectivity the idiomatic version still wins, because a near-perfectly-predicted branch is almost free while the branchless version always pays for a million writes. It also hurts readability and is easy to get wrong, and modern compilers already autovectorize many such patterns. The practical takeaway: a branch is cheap, a mispredicted branch is not, and this trick is worth reaching for only when a profiler points at a hot loop branching on genuinely unpredictable data.