hn.today

Artificial Analysis Intelligence Index v5

artificialanalysis.ai62 points67 comments
Screenshot of Artificial Analysis Intelligence Index v5

Artificial Analysis Intelligence Index v5 is a revised comparison of AI models across nine evaluations intended to reflect practical knowledge work, coding, science, and general reasoning. It adds private real-world coding and scientific research tests, replaces Humanity’s Last Exam with a refined 1,000-question set, and updates its knowledge and physics evaluations. Private test data or held-out answers now account for 60% of the Index, reducing the risk that models benefit from exposure to public questions. The four categories receive equal weight. The accompanying Progress Index offers a separate scale for tracking capability changes across models released since November 2022.

Claude Opus 5.5 leads the overall ranking with 58 points, followed by GPT-6 Astra at 57 and GPT-6.1 Sol at 55; no model tops all nine evaluations. Open-weight models lag further behind than in the previous version: Kimi K3.1 Pro scores 46, while the strongest models with released weights score 32. Performance also comes with substantial differences in cost: GPT-6.1 Sol is within four points of Opus at roughly one-eighth its per-task cost. Higher effort settings can raise scores but sharply increase expense, and the expanded agentic tasks demand more output tokens. The results show strong progress at the frontier, alongside substantial room to improve across every evaluation and particularly large gaps in scientific research and difficult knowledge tests.

Read on artificialanalysis.ai67 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.