hn.today

Gemini 4 Neon looks like an Opus 5.5 competitor

micjm.com253 points78 comments
Screenshot of Gemini 4 Neon looks like an Opus 5.5 competitor

The piece evaluates whether Gemini 4 Neon could compete with Opus 5.5 on the Agent Duel leaderboard. Gemini 4 Carbon led the leaderboard when early votes were excluded, but its main drawback was speed: it generated 46 tokens per second. Neon was observed at 127 tokens per second through OpenRouter, alongside incremental benchmark improvements. To estimate how much that speed matters, the author modeled leaderboard win rates using model identity, generation speed, and whether a user voted early. Adding an interaction between speed and early voting improved the model’s fit, and the results indicate speed is almost as influential as model identity.

Assuming Neon performs about as well as Carbon - a reasonable assumption given Neon’s stronger benchmark scores - and maintains its observed speed, the model projects a leaderboard Elo of 1,066. That is close to Opus 5.5’s 1,078 in a later analysis snapshot. Neon’s estimated win rate against Opus is 48.3%, counting ties as half a win; the 90% interval for the Elo difference runs from 94 points behind to 68 points ahead. The projection therefore places Neon roughly level with Opus, rather than establishing it as the superior model. It depends on modeled leaderboard effects and assumed Carbon-like performance, not a direct comparison between the two models.

Read on micjm.com78 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Mapping every neuron in Qwen3-8B

Mapping every neuron in Qwen3-8B

Researchers found individual neurons in Qwen3-8B that reliably steer its replies into distinct personas or styles, including pirate, German-speaking, and childlike voices. Nudging these neurons changed outputs across many questions without prompting or fine-tuning, revealing how specific behaviors can be localized and edited inside a language model. (fermionresearch.com)

ChatGPT for SNES

ChatGPT for SNES

ChatGPT for SNES runs a 2.2-million-parameter language model on the Super Nintendo’s Super FX 2 chip, with an optional Wikipedia fact book. The unofficial experiment can run in an emulator or on a flash-cart-equipped console, but its tiny model often gives incorrect answers. (bxfoundry.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.