hn.today

GPT-6.2 Astra scores 100% on ARC-AGI-3

twitter.com38 points14 comments
Screenshot of GPT-6.2 Astra scores 100% on ARC-AGI-3

ARC Prize reports that OpenAI’s GPT-6.2 Astra is the first model to achieve a perfect score on ARC-AGI-3, with the result verified by the benchmark organization. At maximum reasoning, it scored 100% using the provider-adapter setup, at a reported cost of $4,100; in the standard setup, it scored 70.3% at $24,200. Astra also set a reported state-of-the-art score of 95.8% on ARC-AGI-2, costing $0.50 per task, and scored 98.5% on ARC-AGI-1 at $0.17 per task.

The accompanying comparisons give context for the ARC-AGI-3 result. Sol’s best score was 98.2% at high reasoning, costing $1,500 with the adapter - 1.8 percentage points below Astra, but 64% cheaper; its standard score was 54.9% at $5,500. Across 25 public ARC-AGI-3 games at maximum reasoning, Astra used 64.1% fewer actions than the human baseline and 5.2% fewer than GPT-6 Astra with the adapter. The reported efficiency comparisons also say GPT-6.2 Astra used fewer reasoning tokens than its predecessor on ARC-AGI-2 at most effort levels, and 53% fewer first-action reasoning tokens on semi-private ARC-AGI-3 games across effort levels.

Read on twitter.com14 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.