Bot floods force Wayback Machine to tighten rate limits, snaring some real users

The Internet Archive says its Wayback Machine has been buckling under repeated surges of high-volume automated traffic, prompting the organization to deploy protective measures to keep the service online. Among the recent changes is a reworded 429 “Too Many Requests” response, the HTTP status returned when the archive throttles a client for exceeding its limits.

The tradeoff is collateral damage: the same defenses occasionally block legitimate visitors who aren’t behaving like abusive bots. The Archive acknowledges the false positives and says it is refining its ability to distinguish scrapers from the everyday users who rely on the archive.

Anyone who believes they were blocked in error can email [email protected] with their operating system, browser, and IP address for a manual review. The episode underscores the operational strain that automated crawling places on free, public-good infrastructure, and the blunt-instrument nature of rate limiting as a defense.