hn.today

Why is Gemini 4 Carbon taking so long?

kernel.dev229 points120 comments
Screenshot of Why is Gemini 4 Carbon taking so long?

A benchmark of difficult bug-fixing tasks compared Gemini 4 Carbon with GPT-6 Astra and Opus. Each model received a codebase containing an unfixed bug and a symptom-only report, then had to identify and repair the cause without unrelated changes. Carbon sometimes took close to an hour on tasks that the other models completed in minutes. Slower inference explained 16-30% of its extra time versus Astra and 45-49% versus Opus; most of the gap came from Carbon doing far more work. Per bug, it used about seven times as many steps and eleven times as many tool actions as Astra, wrote seven times as much, and processed over sixteen times as much input.

Trace analysis found that roughly half of Carbon’s steps addressed issues beyond the requested fix: it continued auditing related code after finding the cause and guarded against scenarios the report had already ruled out. In a sample run, Carbon identified the cause after about six minutes but waited another 25 minutes before editing. Regular prompts asking whether it had enough information to fix the bug shortened the run from 37 to 28 minutes, suggesting the delay reflected its tendency to overwork rather than a failure to understand the problem. Carbon also made fewer tool calls per step and reread 39% of the lines it inspected, compared with 7% for Astra. These mechanical inefficiencies account for an estimated quarter of its excess runtime.

Read on kernel.dev120 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.