LLMs Invent Their Own Social Biases — and Bigger Models Do It Worse

Researchers at Princeton and the University of Chicago show that language models don’t just echo human prejudice — they manufacture fresh biases on their own. Using a decision-making paradigm borrowed from psychology, the team had models repeatedly assign tasks to members of invented demographic groups that were, by construction, identical. The models nonetheless drifted toward sharply stratified allocations, favoring some fabricated groups over others despite the absence of any real difference. The resulting assignments were less equitable than those made by human participants given the same setup, and the effect grew stronger, not weaker, in newer and larger models.

The authors trace the behavior to a botched exploration-exploitation trade-off. A model samples too few examples from each group before settling on a verdict, so a handful of early, noise-driven observations calcify into a durable impression of an entire population. In other words, the bias is a byproduct of how the systems learn from experience on the fly, not a residue of biased training text. That framing matters as LLMs are increasingly wired into agentic pipelines that make consequential, repeated choices about people — hiring triage, resource routing, moderation — where self-reinforcing snap judgments could compound quietly.

Interventions aimed at inputs, problem framing, and explicit steering mostly failed. The only reliable fix was directly incentivizing the model to keep exploring before committing, which curbed the stratification. The finding pushes back on the comfortable idea that models are passive mirrors of society’s existing biases: left to act, they can originate new ones, which raises pointed governance questions about auditing and constraining autonomous AI decision-makers over time.