The Slop Index benchmarks how much AI-generated writing resembles stereotypical chatbot prose rather than ordinary human writing. It compares 58 frontier models across 112 scenarios and 64,636 generations, with higher scores indicating more “slop.” The benchmark’s central claim is that recognizable AI style can be measured rather than left to intuition: common signals include formulaic greetings, neatly packaged points, stock phrases, and overly polished rhythms. It contrasts a deferential, generic email with a brief, specific message about delaying work, illustrating the difference it aims to capture.
The score combines mechanical measurements with blind human judgments. Forty percent comes from votes in an arena where people compare two anonymous samples written for the same task; the remaining 60 percent is divided equally among conciseness, templating, rhythm, and stylistic tells. These signals are evaluated against examples of human writing that predate generative AI, including workplace emails from 2000-2002, pre-2022 tweets, essays, and chat logs, so the reference standard was not shaped by ChatGPT. The project presents its leaderboard as live and offers a public method and GitHub materials for inspecting or reproducing the measurements. The provided information describes the benchmark’s design but does not give the models’ scores or rankings.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.