A benchmark report compares how modern large models perform when asked to use a browser like a human, measuring accuracy, latency (seconds), and cost per task across different harnesses (integration setups) and providers. The central claim is that the harness that drives an agent - how prompts, tools, and orchestration are arranged - shapes outcomes as much as the underlying model. Results are presented as ranked runs (15 shown of 70 total), and the SDK for running these browser-agent experiments is provided so teams can reproduce or add their own models and harnesses.
The numeric findings illustrate the claim: top accuracy was 89.0% for claude-opus-5-5 on the "claude code" harness (326s, $0.51), followed by gpt-6-astra on "codex" at 87.0% (320s, $3.07) and claude-fable-5-1 on "claude code" at 84.0% (552s, $1.63). The same model shows wide variation by harness: claude-opus-5 ranges from 78% accuracy (678s, $2.44) to 71% with an "fx" harness that costs $10.02 and runs 735s. Other examples highlight tradeoffs: gpt-6-sol hits 76% in 268s for $0.41, while gpt-6-luna posts 74% for only $0.0225. The dataset makes clear that integration and orchestration choices materially alter accuracy, speed, and cost.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Rails is optimized for AI development by providing clear conventions, efficient code, and a full-stack framework for solo developers. Benchmarks show varying model performances in accuracy, speed, and cost when executing Rails-related tasks. (rubyonrails.org)
Vibe Coding Production Kit is a workflow designed for developing AI coding agents, streamlining the process from issue identification to code deployment. It integrates with tools like GitHub Copilot and offers resources for developers working on AI-powered software development. (github.com)
SkillOpt treats natural-language skills as trainable external artifacts for frozen agents, optimizing them through rollouts, reflection, and validation. It improves agent performance by making the skill document itself the target of structured edits, avoiding fine-tuning of the underlying model. (microsoft.github.io)
Opus 5.5 can generate explainer videos automatically using a serverless agent on OpenComputer, taking a URL or prompt to produce a video in about four minutes. It leverages a microVM environment, web tools, and a deterministic rendering process to create high-quality videos without manual editing. (launchvideo.io)
Anthropic has signed a $12 billion agreement with Akamai to access AI computing resources. The deal aims to support Anthropic's development of advanced artificial intelligence models. (bloomberg.com)
Tech leaders from major AI companies have urged the United Nations Security Council to regulate artificial intelligence technology to prevent potential threats to humanity. They warn that unchecked AI development could pose significant risks to global safety. (apnews.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.