AI Slop Threatens Trust in Science

Generative AI can now produce scientific papers that look credible with almost no human effort, creating a growing challenge for the people and systems responsible for protecting the scientific record. The central concern is not simply that AI makes mistakes, but that fabricated or low-quality material can become difficult to distinguish from genuine research as language models improve. The familiar advice to “trust but verify” becomes much harder to apply when publishers must check a flood of submissions rather than individual chatbot answers.

An experiment by mathematician Benjamin Skuse tested whether AI-generated papers could pass IEEE’s publishing safeguards. He found that creating and submitting a roughly plausible paper could require little more than running an experiment and prompting an AI to write it. IEEE’s human reviewers and research-integrity software currently appear able to catch many such attempts, but increasingly capable models may evade the errors that expose output from free tools. This puts publishers in an ongoing arms race. Wikipedia volunteers face a similar task identifying and removing machine-generated material, with fewer resources. If scientific findings and basic reference information can no longer be trusted, confidence in institutions that curate knowledge - and in the possibility of a shared understanding of reality - may erode. The prospect of AI systems producing work primarily for other AI systems raises a further question: whether people could eventually be excluded from creating and evaluating knowledge.