hn.today

The Slop Index: Which AI Model Writes the Least Like AI?

theslopindex.com9 points1 comments
Screenshot of The Slop Index: Which AI Model Writes the Least Like AI?

The Slop Index benchmarks how much AI-generated writing resembles stereotypical chatbot prose rather than ordinary human writing. It compares 58 frontier models across 112 scenarios and 64,636 generations, with higher scores indicating more “slop.” The benchmark’s central claim is that recognizable AI style can be measured rather than left to intuition: common signals include formulaic greetings, neatly packaged points, stock phrases, and overly polished rhythms. It contrasts a deferential, generic email with a brief, specific message about delaying work, illustrating the difference it aims to capture.

The score combines mechanical measurements with blind human judgments. Forty percent comes from votes in an arena where people compare two anonymous samples written for the same task; the remaining 60 percent is divided equally among conciseness, templating, rhythm, and stylistic tells. These signals are evaluated against examples of human writing that predate generative AI, including workplace emails from 2000-2002, pre-2022 tweets, essays, and chat logs, so the reference standard was not shaped by ChatGPT. The project presents its leaderboard as live and offers a public method and GitHub materials for inspecting or reproducing the measurements. The provided information describes the benchmark’s design but does not give the models’ scores or rankings.

Read on theslopindex.com1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Now Anthropic is blocking Russian users

Now Anthropic is blocking Russian users

Anthropic is accused of charging Russian Claude users before identity checks, then cutting off access after verification and retaining biometric data. Critics say account bans can affect linked GitHub projects and Russian citizens living abroad, and accuse the company of failing to answer public questions. (twitter.com)

The End of AI Frontiers

The End of AI Frontiers

Microsoft Research’s AI Frontiers group brought together about 50 researchers to develop agent systems and foundational AI advances. After the team joined Microsoft AI in September, its research-first approach clashed with leadership’s emphasis on scaling. (hunch.net)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.