hn.today

Stupid Over-Reliance on Palantir AI Helped Lead to US Bombing of Iranian School

techdirt.com28 points1 comments
Screenshot of Stupid Over-Reliance on Palantir AI Helped Lead to US Bombing of Iranian School

US forces carried out a strike on an Iranian elementary school that killed well over 100 children after the site, long used as a military compound but clearly showing playgrounds and painted walls in later imagery, remained misclassified as an IRGC facility. Decision-makers relied heavily on Palantir’s Maven system, which fuses more than 150 data inputs and had been pushed to the center of the targeting “kill chain.” Maven surfaced the Minab site as a recommended day-one target and compressed what used to be hours of human analysis into minutes; no civilian-harm-mitigation (CHM) reviewer examined the site before the strike. Palantir insists its software is not to blame for underlying data or analytic gaps, and former military officials now work closely with the vendor inside command centers.

The chain of failure combined oversold confidence in AI with organizational choices that gutted safeguards: a CHM workforce was slashed by roughly 90% and largely excluded from planning, and commanders expected the automated system to flag stale or inconsistent intelligence - expectations Maven was not designed to meet. The result was not a spectacular runaway AI but mundane human complacency and haste enabled by a tool marketed as “intelligent.” The criticism is that selling automation as a substitute for careful, slow checks turned a preventable classification error into a devastating, arguably unlawful, strike.

Read on techdirt.com1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Security

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.