Flock Safety can’t seem to stay out of the news recently, from the latest wave of attacks on Flock surveillance cameras to claims that Flock’s gunshot detection microphones are also picking up private home conversations. But one specific report has caught my attention — a study showing a startlingly high failure rate for the surveillance system AI.

Read more: Everything You Should Know About Flock Surveillance and How it Works

Following up on complaints about Flock Safety flagging vehicles incorrectly, the police department of Roseville, California, ran its own analysis of the cameras, studying nearly 1,500 Flock reports. They found an astonishing 71% failure rate, where the cameras would mistake one license plate number for another.

That’s not just a snafu. Mistaking one number for another, like seeing a 7 as a 1, can lead Flock cameras to incorrectly identify an innocent vehicle as one that was previously stolen. That’s exactly what happened in the California situation, where a citizen complained that the cameras were repeatedly flagging his car to police as stolen because the AI technology interpreted a 9 as an 8.

This is a stark contrast to Flock’s own claims — sort of. The company reports that independent evaluations show its cameras have a 96% vehicle count accuracy rating, while correctly categorizing 92% into vehicle types. But accurately recognizing vehicles is a far cry from accurately reading license plates, which Flock doesn’t provide much data on. I’ve reached out to Flock to learn more, and I’ll let you know if the company has additional details to provide.

Why this Flock failure rate could cause trouble

It’s important to note that this analysis was conducted between 2023 and 2024. Many AI Large Language Models can improve over time with better training and more access to material, which is one reason AI companies are literally devouring old books these days. So Flock’s cameras may have improved in accuracy by now. However, we’ve also seen the opposite happen: AI models can eat up bad data, especially “recursive” data that was created by another AI, and start becoming dumber, less accurate when performing their tasks.

Flock has said that the camera setup Roseville used wasn’t typical, but didn’t provide more details. It’s possible that where cameras are placed and when they are activated (at night, in the rain, etc.) can have a significant impact on accuracy. So can license plate covers, dirty license plates and unusual license plate placement.

The problem is these are issues any city may face when adopting Flock cameras. Flock’s primary value offering is that the company’s surveillance is ubiquitous, speedy and accurate, doing a job that police can’t. But if the cameras make mistakes when encountering everyday situations, that’s more work for law enforcement, not less.

Cities, including Los Angeles, continue to cancel Flock contracts (although sometimes they just switch to another provider), often due to public pressure. If Flock’s core services could have such a high failure rate, that’s another reason for police and sheriff departments to reconsider their use, and could make it difficult for Flock to sell new cities on its technology.

Tyler Lacoma

Editor / Home Security and Smart Home

Tyler has worked on, lived with and tested all types of smart home and security technology for over a dozen years, explaining the latest features, privacy tricks, and top recommendations. With degrees in Business Management, Literature and Technical Writing, Tyler takes every opportunity to play with the latest AI technology, push smart devices to their limits and occasionally throw cameras off his roof, all to find the best devices to trust in your life. He always checks with the renters (and pets) in his life to see what smart products can work for everyone, in every living situation. Living in beautiful Bend, Oregon gives Tyler plenty of opportunities to test the latest tech in every kind of weather and temperature. But when not at work, he can be found hiking the trails, trying out a new food recipe for his loved ones, keeping up on his favorite reading, or gaming with good friends. See full bio