False Facial Recognition Leads to Wrongful Arrest for Child Kidnapping

What Happened?

Robert Dillon, a Florida resident, was arrested for attempted kidnapping of a minor in a city he had never visited. The only evidence linking him to the crime was a facial recognition match performed by police software, which indicated a 93% similarity with the suspect captured on security video. However, the image used was a low-resolution photograph taken with a mobile phone from a camera screen, with the face partially shaded and at an angle. According to the lawsuit filed by the ACLU (PDF), the officer who responded to the scene did not obtain a direct copy of the video but took photos of the screen with his cell phone. Those grainy second-hand images were then entered into the facial recognition system of the Jacksonville County Sheriff, despite the quality being manifestly insufficient for reliable identification. The system, operated by the sheriff's investigator, returned a 93% confidence that the suspect was Dillon, and based on that result, an arrest warrant was obtained.

Why Is This Important?

This case highlights the serious risks of relying on facial recognition as the sole evidence for obtaining arrest warrants. The county sheriff himself, T.K. Waters, admitted that a facial recognition "hit" is not enough to establish probable cause. In statements to local media, Waters said: "If you come to me with a facial recognition result and that's your probable cause, I'd probably kick you out of my office because that's not how things work." Yet in practice, his office acted precisely that way: the investigator used the facial match as the basis to recommend the arrest. The ACLU lawsuit underscores that the system's accuracy critically depends on image quality, and using low-quality images can lead to false identifications. The case adds to a growing list of facial recognition errors that have disproportionately affected people of color, although in this case Dillon is white. A 2019 study by the National Institute of Standards and Technology (NIST) found that many facial recognition systems have higher error rates for people of certain demographic groups, but the fundamental error here is the reliance on low-quality images, a problem that affects everyone equally.

What Consequences Will It Have?

The case could set a legal precedent on the use of artificial intelligence in police investigations. If the lawsuit succeeds, it could force agencies to adopt stricter standards for facial recognition use, such as requiring high-quality images and corroboration with other evidence. Additionally, it reinforces calls for a moratorium on police use of this technology until proper safeguards are established. Cities like San Francisco, Boston, and Portland have already banned police use of facial recognition, while others like New York have imposed restrictions. This case could accelerate those initiatives at the state level. It could also influence federal legislation: currently, Congress is debating several proposals to regulate AI, including the Algorithmic Accountability Act. The Dillon case vividly illustrates why clear rules are needed. Furthermore, the lawsuit could result in compensation for Dillon and changes in Jacksonville Beach police protocols. If officers are found to have acted negligently, they could face disciplinary sanctions and mandatory training on the limits of AI.

What Should Readers Know?

• Facial recognition is not infallible, especially with low-quality images. According to the ACLU, the system's accuracy "depends significantly on the quality of the test image; lower-quality images contain less interpretable facial data, degrading the system's ability to produce a reliable template."
• Police should not base an arrest warrant solely on a facial recognition result. Sheriff Waters implicitly acknowledged this, but his office acted to the contrary.
• The ACLU is suing the city of Jacksonville Beach, Sheriff T.K. Waters, and several officers for civil rights violations, including false arrest and due process violations.
• This case adds to other incidents of AI misidentification, such as that of Robert Williams in Detroit, who was wrongfully arrested in 2020 after facial recognition identified him as a watch thief. Williams spent 30 hours in jail before DNA evidence exonerated him. In 2022, another man, Michael Oliver, was unjustly arrested in New Jersey for a similar error. These cases show a pattern of over-reliance on technology without proper verification.
• Facial recognition technology has a history of racial bias. A 2018 MIT Media Lab study found that systems from Microsoft, IBM, and Face++ had error rates of up to 35% for dark-skinned women, compared to less than 1% for light-skinned men. Although Dillon is white, the systemic problem persists.
• The lawsuit seeks to declare current practices unconstitutional and impose corrective measures, such as requiring that any facial recognition identification be corroborated by independent evidence before seeking an arrest warrant.

"If you come to me with a facial recognition result and that's your probable cause, I'd probably kick you out of my office because that's not how things work," said Sheriff T.K. Waters.

This case is a reminder that AI is a tool, not a substitute for human judgment. Police must use facial recognition with caution, ensuring images are high-quality and results are verified with other evidence. Otherwise, there is a risk of arresting innocent people, eroding public trust in the justice system. As the ACLU noted, "Technology should serve justice, not undermine it."