Artificial Analysis Intelligence Index v5.0

Artificial Analysis Intelligence Index v5.0 is a comparative benchmark for language models, combining nine evaluations of reasoning, knowledge, mathematics, programming, and practical task performance. It assigns equal weight to Knowledge Work, Coding, Science, and General capabilities (25% each). The individual tests measure such abilities as producing project deliverables, automating software workflows, changing code in repositories, completing terminal-based tasks, solving scientific problems, and answering broad knowledge questions. The highest-weighted tests are AA-Briefcase, FrontierCode 2.0, Terminal-Bench-Science, and AA-Omniscience v1.1, each at 15%; the remaining tests carry 5% or 10%. Scoring varies by task, using methods such as test-suite success, rubric-based grading, pairwise model comparisons, and separate measures of accuracy and hallucination.

The methodology prioritizes standardized conditions, zero-shot instructions, answer validation intended to avoid unfair penalties, and disclosure of prompts and scoring rules. Most evaluations use pass-at-one scoring, though some repeat tasks and aggregate results; models are tested with different temperature settings for reasoning and non-reasoning systems. The index primarily assesses English text, while multilingual, visual, speech, and other capabilities are measured separately. Additional benchmarks cover professional and legal work, enterprise operations, coding, medicine, and multilingual performance, among other areas, but do not affect the index score. The framework is designed as a broad comparison rather than a guarantee of performance in any particular use case.