hn.today

GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains

newsletter.semianalysis.com6 points0 comments
Screenshot of GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains

GDPx is a benchmark designed to measure frontier AI agents in domains expected to produce especially high returns from advanced intelligence, rather than sampling work in proportion to its share of the economy. It covers chip and hardware design, cybersecurity, biology, finance, legal work, quantitative trading, recursive self-improvement, materials science, alignment, and software engineering. Its creators argue that AI’s economic impact will be concentrated in such areas, so benchmarks based on broad occupational categories can obscure progress where frontier models matter most. Results show substantial variation by domain: Opus 5.5 leads overall and excels at quantitative trading, hardware, and chip design; GPT-6 Astra leads in recursive self-improvement and materials science; Gemini 4 Carbon is strongest in biology and narrowly leads software engineering. Finance and legal tasks defeat nearly all models. The best model scores only 12% overall, and safety filters depress some scores; excluding cyber, biology, and alignment changes the rankings.

The benchmark’s central design choice is to evaluate agents in realistic, interactive work environments rather than asking them to produce deliverables from context supplied in a prompt. Its finance tasks, for example, require navigating thousands of anonymized deal files; other domains use live services, simulated laboratories, or commercial design software and hardware. This contrasts with GDPval, which the creators criticize for shallow, one-shot tasks, noisy or flawed grading, and a lack of work environments. Because difficult, fair tasks require specialized expertise, GDPx was built with 14 domain-specific partners and subjected to quality review. The reported rankings also show meaningful capability “spikiness” across models and a cost-performance pattern in which lower-cost models compete at the low end, OpenAI occupies the middle, and Opus leads the high end.

Read on newsletter.semianalysis.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Inkling 2 Small

Inkling 2 Small

Thinking Machines introduced Inkling 2 Small, a 594-billion-parameter mixture-of-experts model with image input and stronger coding, vision, reasoning, and autonomous-work capabilities. It is available through Tinker with closed weights, with Apache 2.0 weights planned after safety testing. (thinkingmachines.ai)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.