AI hallucination nearly triggers US Military operation

U.S. military planners nearly launched an armed strike against a Chinese vessel after intelligence compiled with the help of an AI chatbot falsely identified the ship as carrying components for a nuclear weapons program. Aircraft were already airborne when commanders discovered the underlying report was flawed: a Special Operations analyst had asked a chatbot to synthesize open-source material with classified signals intelligence, and the model hallucinated the ship’s cargo manifest. The analyst then used the tool a second time to turn the erroneous result into an official-looking summary that propagated through command channels before the operation was halted at the last minute, averting a potential international incident during ongoing hostilities with Iran.

The episode exposes a critical tension as the Pentagon pushes to integrate AI to speed decision-making and compress the kill chain: the same rapidity that makes large language models appealing also lets fabricated outputs travel up the chain of command if human oversight is insufficient. Experts argue the event is a warning not to abandon AI but to strengthen safeguards, training and verification for any workflow tied to targeting, intelligence analysis or operations that could use force. Veterans and researchers emphasize that personnel must grasp LLM uncertainty, and that adoption without robust checks will produce incidents that erode trust and ultimately hamper effective use.