hn.today

DeepSeek v4.1 Flash Is Now Our Best Hacking Model

enclave.ai144 points57 comments
Screenshot of DeepSeek v4.1 Flash Is Now Our Best Hacking Model

DeepSeek v4.1 Flash achieved verified code execution on all 11 vulnerable targets while four fixed controls remained secure, with accepted runs costing $4.65 (complete runs $5.14). The model performed extensive automated reconnaissance and interaction - 2,349 Bash commands, about 2 hours 38 minutes of active time, 268.3M input tokens (mostly cached) and ~2M output tokens - producing median successful runs of 4 minutes 38 seconds. An audit of every command found six solutions that followed the planned vulnerabilities and five additional successful routes that the original scoring did not distinguish. Notable specifics: Grafana was compromised in under 90 seconds by loading executable files from a temporary plugin folder rather than the intended file-path exploit; Jenkins was breached via a file-pointer trick to read a controller credential and then by using the built-in script console for code execution, plus a separate upload-race run that required precise timing; Nextcloud was exploited by reusing an approved read access to perform a write and overwrite a PHP template.

The outcome-based score remains 11/11, but the path-level review exposed the need to judge attack paths as well as results. The five extra routes were artifacts of the private benchmark environment, not claims of new upstream vulnerabilities; those shortcuts have been closed and checks tightened. Challenge source versions were updated and leaderboard comparisons will require new runs on the revised benchmark. DeepSeek’s runs both demonstrated strong exploit capability and prompted concrete improvements to benchmark design and scoring.

Read on enclave.ai57 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Security

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.