hn.today

Gemini 4 Neon is the new #1 on the Models Engineering Index

git-aarvin.github.io373 points244 comments
Screenshot of Gemini 4 Neon is the new #1 on the Models Engineering Index

The Models Engineering Index (MEI) compares language models on practical engineering work, combining results across benchmarks that include CursorBench, FrontierCode, Migration, IOI, ProgramBench, Terminal-Bench, and Vibe Code. Google’s 4 Neon ranks first with an index score of 69.5, ahead of Anthropic’s Fable 5.2 at 65.8 and Opus 5.5 at 63.1. OpenAI’s 6.2 Astra places fourth at 60.5, while Google’s 4 Hydrogen and Anthropic’s Sonnet 5.5 tie at 57.2. The index is intended to show performance on engineering tasks rather than provide a general-purpose ranking of language models.

The rankings also present cost and token use as separate considerations, making clear that a higher score can come with substantially greater resource demands. Relative figures show Fable 5.2 at 6.5 times the listed cost and 2.1 times the tokens of the comparison baseline, versus 4 Neon at 1.9 times the cost and baseline-level token use. Astra scores below the top three but uses only 0.29 times as many tokens; 6.2 Sol scores 56.9 while listing cost and token use at 0.4 and 0.35 times. Lower-ranked models include inexpensive options such as 6 Luna and V4.1 Flash, whose scores are 34.8 and 44.1. The underlying data is attributed to Cognition, Cursor, the Laude Institute, and Vals AI.

Read on git-aarvin.github.io244 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.