A 16-year-old student in Baltimore was handcuffed by armed police officers after an artificial intelligence system incorrectly identified a crisp packet in his pocket as a firearm. The incident has raised concerns about the reliability and implementation of AI-powered security technology in schools.
Key Takeaways
An AI weapon detection system flagged a crisp packet as a gun.
Armed police responded to the school and handcuffed the student.
The AI provider stated the system "operated as designed" with human verification.
Calls have been made to review the school's AI security procedures.
The Incident Unfolds
Taki Allen, a student at a Baltimore school, was leaving football practice when armed police arrived. He recounted being told to get on his knees and was subsequently handcuffed. Allen explained that he had finished a bag of Doritos and placed the empty packet in his pocket shortly before the police intervention.
Baltimore County Police Department confirmed that officers responded to an alert and that the student was handcuffed but not arrested. A statement indicated the situation was "safely resolved after it was determined there was no threat."
AI System's Role and School's Response
The AI system, provided by Omnilert, initially detected what appeared to be a firearm. An image was sent to human reviewers who found no threat. However, the school's principal, Kate Smith, reportedly missed this cancellation and contacted the school's safety team, who then alerted a school resource officer. This ultimately led to the police being called.
In a letter to parents, Principal Smith stated that the school's safety team had "quickly reviewed and cancelled the initial alert after confirming there was no weapon." She added that the school resource officer contacted the local precinct for additional support, and responding officers confirmed the absence of any weapons.
Provider's Statement and Broader Concerns
Omnilert, the AI provider, expressed regret over the incident and stated that its system "operated as designed" by flagging a potential threat for rapid human verification. The company noted that "real-world gun detection is messy" and emphasized the process's intent to "prioritise safety and awareness through rapid human verification."
However, the incident has prompted local politicians, such as Baltimore County councilman Izzy Pakota, to call for a review of the school district's procedures regarding its AI-powered weapon detection system. This event echoes previous scrutiny over the accuracy of AI in identifying weapons, with a past instance involving another company, Evolv Technology, facing bans for unsubstantiated claims about its AI scanners used in public spaces.
Sources
Armed police handcuff teen after AI mistakes crisp packet for gun in US, Yahoo News Canada.