Surveillance in public urban spaces is becoming more and more ubiquitous, whether or not people are aware of it. Thanks to advances in computer vision technology, the cameras that are watching people on the New York City streets are now counting and tracking them, too.
Real-time pedestrian data is extremely valuable to city planners, business owners, police interested in crowd control, and New York City’s residents as well, who might want to know, say, how long the line is for a cronut before they commit.
An eight-person startup called Placemeter is working to cobble together video feeds from different sources and cover all or most of New York City’s sidewalks and public spaces within the next year or so. Founder Alexandre Winter estimates Placemeter needs about 2,000 to 3,000 well-placed cameras to cover 90% of the city, and so far the company has access to about 500. Mostly, these are from a partnership with New York City to more easily access its publicly available traffic camera feeds as well as its work with live webcam sites like Earthcam.
As you can see above, Placemeter’s algorithms can recognize both vehicles and pedestrians walking down the street. By also recognizing when a person enters or exits a building, it can estimate the size of a crowd inside a building or storefront at any given time. It can also track when the same individual walks from one building to the next—information that competing retailers would love to know. Winter says its counts are 80% to 95% accurate.
The firm does not store any data or video on servers after it is analyzed, so it can’t know if the same people walk down the same street every day or say respond to a court subpoena, Winter says. It has made it a policy to not attempt to identify individuals or use facial recognition, but it is not necessarily a technological limitation. "By construct, the system is completely privacy safe," says Winter. "We don’t identify people at all."
Even so, it’s a good example of what is possible today, and especially with higher resolution cameras and drones becoming cheaper, and it’s not hard to imagine how closer tracking would be possible. Already, around the city, New York is installing license plate readers to track the movements of cars in and out of Manhattan.
Winter says the beneficial uses of Placemeter’s street data are enormous. "Cities are more and more becoming the places where people live," he says. "We have a real need to optimize how people use their cities." It already has provided low-resolution analysis of pedestrian counts by neighborhood for New York City’s new Business Atlas, an interactive map of city data that is intended to help businesses plan their siting and marketing strategies, and will update that data about once a month. The city might see other uses, he says, such as police wanting to know about crowds, such as at a club, parade route, or demonstration. The company plans to sell its much street-by-street, real-time data to business customers and eventually create a consumer app, so New Yorkers could spend a little less of their time waiting in lines and crowds.
For now, the focus is more about gathering all the data it needs and making sure it has the server power to process the data, given that each camera in its existing network recognizes 10,000 to 100,000 objects per day. "We can get there, but that’s a lot of data to process and a big challenge for us is scalability." The company, which originally participated in the Techstars incubator program, is still small.
Its plan to add more cameras to its network is interesting. Like the service Waze (now owned by Google) crowdsources real-time traffic reporter from people stuck in jams, Placemeter wants to develop a small network of contributors who will mount their unused smartphones to their windows and videotape their streets. It has just developed a Sensing app that people could install on their phone through Wi-Fi (no carrier service needed) that would analyze the video on the device and transmit the data back to Placemeter. It even wants to provide a window mount.
Winter says consumers could see benefits back from the crowdsourced video feeds, if Placemeter develops a line-waiting app or an app that sends alerts when it thinks the local grocery store isn’t crowded or a basketball court is free. In residential areas, he’s seen some interest from neighborhood associations who are lobbying to get more trash cans installed on streets or trying to prove the speed limit needs to be changed in their neighborhood (people could even probably hack together their own speed cameras with the technology).
Image recognition and analysis isn’t the only way to estimate the crowds or flow of people. Other companies are directly buying anonymous data from wireless carriers and GPS providers (which is expensive), or installing devices that detect Wi-Fi pinging signals on people's smartphones as they walk by. But analyzing video is on the only way to get a comprehensive count down to the level of individual addresses (their data is accurate to within a few square meters). "The reason why these technologies are picking up now is first the data is available," Winter says.