吃瓜大本营

A face recognition and video analytics company has that provides a stark reminder of the power of these technologies and how they are likely to be used over time by law enforcement, powerful corporations, and others, if we as a society allow it.

The technology in question involves video search, which we described in our 2019 video analytics report. In the past, video operators looking for something would have to manually scroll through many hours of video, but technology is increasingly automating such searches. In a presentation for subscribers of the surveillance research group IPVM, a company called Vintra presented on its technology for quickly searching through large stores of video footage

The relevant of the is worth watching. In it, a company executive searches through a month鈥檚 worth of video footage captured by around 10 fixed cameras, plus body cameras, in a transit center in San Jose, California. He feeds the system with the photograph of a male subject, and the system does a face recognition search through all the stored video from that month and produces 23 snapshots of the man from the center鈥檚 cameras. Clicking on any of the snapshots plays the video in which he was captured.

Already, that鈥檚 a demonstration of the stunning new power that surveillance camera systems create when combined with face recognition and today鈥檚 search capabilities.

But there鈥檚 more. The Vintra executive then presses a button called 鈥淔ind associates.鈥� He selects a time period 鈥� he uses 10 minutes but it could have been shorter or longer 鈥� and then runs a new search. This search yields snapshots of 154 other people, each of whom was seen on camera within 10 minutes of the subject.

In other words, this system allows face recognition to be used to track not just one person, but to map out people鈥檚 associations with each other.

Of the people spotted with the subject in the demo, 150 appeared on camera with him only once, and another three appeared with him twice. One man, however, had 14 鈥渃o-appearances鈥� with the subject 鈥� clearly not coincidence, but a result of some association between the two men. The system displayed snapshots of the 14 co-appearances, and clicking on them instantly played the video of the two of them together.

The men could be anything from co-workers to commuting partners to lovers. Perhaps clicking through to view their joint appearances would shed light on which. But whatever the case, their association has now been revealed to the prying eyes of this camera network and its operators. One of Vintra鈥檚 mottos is 鈥淜now what the cameras know,鈥� and if this lives up to the demo, it鈥檚 a spookily accurate slogan, not least because it captures the way that AI is allowing video cameras to 鈥渨ake up鈥� 鈥� rendering them able not just to dumbly record video, but increasingly to understand what they鈥檙e seeing.

With this kind of technology, as the Vintra pitchman put it, 鈥淵ou can really start building out a network. You may have one guy, that showed up a few times, that you鈥檙e interested in 鈥� you can start looking at windows of time around him to see who else is there at the same time, and build out the networks of those people.鈥�

Too many conversations about surveillance focus on how information could be used in isolation against a specific individual. But analytics is a powerful tool, and when information is collected not about just one suspect, but about large numbers of people, we often forget that such data can be cross-referenced to create maps of associations. I wrote this piece in 2013 to try to hammer home that often non-intuitive point, but maps of people鈥檚 associations (called 鈥溾��) have long been a product of mass surveillance. It has been done by the NSA, the U.S. military overseas wide-area aerial surveillance, for example.

Now, face recognition and other analytic techniques appear to have brought social network analysis to video surveillance. And who knows what purposes such mining could be used for. The Vintra pitchman told his security audience that his product 鈥渨ill plug in to BI tools鈥� 鈥� referring to Business Intelligence, a catch-all buzzword referring to non-security uses of data such as competitive research and marketing: 鈥淵ou may be using the cameras for security, but 94, 96 percent of the time there鈥檚 no event that security鈥檚 interested in 鈥� but there鈥檚 always information that the system is generating on those that you can plug into your BI.鈥�

The bottom line is that when we see a video camera today, we need to update our intuitions about what it鈥檚 capable of. It may no longer be just collecting inert and unused video, but, especially if that camera is part of a larger network, the data it collects could be mined for insights about our lives across space and time. Communities and policymakers considering the installation of surveillance cameras 鈥� especially camera networks 鈥� should take heed.