University libraries are being swarmed by AI bots, putting public access at risk

University libraries, museums and digital archives are facing surges of automated traffic from bots collecting material for AI training. Their open collections - often containing vetted academic, historical and government documents - are attractive because they are accessible without paywalls or logins. Since late 2024, institutions across North America, Europe and Oceania have reported disruptions, and the problem is worsening. Bot traffic can slow or temporarily knock websites offline, leaving students, researchers and members of the public unable to use them. The University of North Carolina at Chapel Hill, for example, recorded more than 11,000 searches for Finnish music in one day, and its catalogue periodically blocked access for campus users.

The central concern is not simply that AI systems gather public material, but that poorly behaved scrapers ignore site instructions, make enormous numbers of requests and can undermine the public access that libraries have worked to provide. Unlike conventional search-engine crawlers, they may request thousands of documents per second and download multiple formats; blocking individual IP addresses is ineffective when traffic comes from many addresses. Libraries have tried CAPTCHAs and login requirements, but these can also exclude legitimate users and helpful bots. Major AI firms have backed a voluntary, standardized opt-out signal, though it would not cover third-party scrapers or smaller companies. Experts warn that disruption may discourage institutions from digitizing and sharing collections, reversing a major public benefit of online access.