In October 2025, four Black high school students waiting for rides after football practice were confronted by police after Omnilert, a school AI gun-detection system, mistook a Doritos bag for a firearm. School security staff reviewed the footage and cancelled the alert within two minutes, but the principal later missed the cancellation and asked a school resource officer to send police. At least six patrol cars arrived; officers reportedly drew guns, ordered the teens to their knees, handcuffed and searched them, and did not explain the reason until they found no weapon. The students say the encounter left them frightened, physically marked by handcuffs, less trusting of police, and anxious about waiting outside school. One now pays for rides to avoid waiting, while another repeatedly calls his mother to check when she will arrive.
The teens are suing the school district, officials, police officers, and Omnilert, alleging unlawful detention and searches and seeking to block the system’s use until safeguards and training are in place. A school inspector general found no bias in the software but documented failures in how alerts were handled: the principal had not been trained, and more than 99 percent of the district’s alerts were false positives requiring no police response. The inspector general recommended regular training, tighter alert distribution, and using the app so cancellations are harder to miss. Omnilert says human review is central to its design and that a cancelled alert must remain cancelled; it apologized for the students’ experience. The case challenges the use of AI in public safety when human judgment and communication fail.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.