Suspicious Discontinuities: How Bright-Line Thresholds Expose Gaming and Fraud

Wherever a sharp threshold carries real consequences, the data around it tends to bend in revealing ways. Dan Luu walks through a series of histograms and scatterplots where a smooth distribution suddenly breaks at a magic number, and each break points to someone responding to an incentive. When universities began treating Pell Grant eligibility as a proxy for serving low-income students, admissions chances jumped at the income threshold — but perversely, among non-qualifiers the poorest applicants were hurt most, and among qualifiers the richest benefited most. Savvy parents even shed income at the last minute to drop below the line, sometimes by deliberately losing money on options trades.

The same fingerprints show up across wildly different domains. Russian election results spike at round turnout numbers like 95%, a signature of fabricated tallies that don’t bother mimicking a natural distribution (the kind of anomaly Benford’s law is built to catch). Psychology papers cluster suspiciously at p-values just under 0.05; U.S. cocaine possession charges piled up at exactly 280 grams after the Fair Sentencing Act moved the mandatory-minimum trigger there; and Polish high-school language exams show a glut of scores at or just above the passing mark of 30, because graders hunt for a missing point to spare a student from failing — an effect absent in math, where partial credit is harder to manufacture.

The throughline is that discretizing a continuous measure into a pass/fail or qualify/don’t-qualify rule reliably distorts behavior near the cutoff, whether through outright fraud, motivated grading, or rational gaming. The youth-soccer example adds a subtler twist: birth-month cutoffs produce a sawtooth in who makes elite leagues, and the players who clear the bar despite being young for their cohort turn out to be more valuable — a relative-age version of the same selection bias seen in studies of discrimination in pro sports. The practical lesson, echoing Andrew Gelman’s campaign against significance thresholds, is that bright-line rules invite manipulation, and these discontinuities are a cheap, powerful tool for spotting it.