In a partnership among Bellevue, Wash.; Microsoft; the University of Washington; and the Institute of Transportation Engineers, researchers are hoping to use object recognition through video analytics to recognize collisions and near-misses on roadways. Called the Video Analytics Towards Vision Zero, the initiative strives to eliminate collisions on roadways around the world by training a program to spot near-misses in intersections and understand the road conditions that led to the outcome.
The project is based on a study that used GoPro cameras to recognize dangerous traffic corridors in Montreal. Bellevue Principal Transportation Planner Franz Loewenherz knew it would be impractical as a wide-scale strategy for municipalities, but he realized there was an entire trove of video streams in intersection cameras.
“What if we could use our existing infrastructure and take raw video footage from those cameras and translate it into real data that we can then make smart decisions on?” he told Government Technology. “We already have hundreds of cameras."
The program looks at various road users, cars, cyclists and pedestrians in incremental clips to understand what their behavior looks like. The more input the system has, the better the program will be at recognizing the different roadway elements. So they opened it up to the public to help teach the algorithm what a pedestrian "looks like" versus a motor vehicle or cyclist.
Anyone can visit the site, watch a short clip, and manually categorize the various objects passing through their frame. Through the collective action, the input builds a baseline knowledge for the computer to automatically determine what it sees more accurately across multiple contexts. In approximately one month, 500 people have volunteered to annotate existing traffic footage via the crowdsourcing platform.
Loewenherz is hoping the increased input via crowdsourcing will help the tool identify dangerous situations and give planners the chance to make proactive changes in order to improve an intersection’s safety before people are hurt.
While the original project began as a relatively small partnership between Microsoft and Bellevue, it was always intended to grow larger and have a bigger impact than just within city limits, explained Loewenherz.
“One of the things that was important to me, both when I did the [Americans with Disabilities Act] work and the work I’m doing now, is that it be transferrable,” he said. “I’d never want this to be just about Bellevue. I wanted this to be available to cities small and large in the U.S. and elsewhere.”
Partners now include New York City, San Francisco, Los Angeles and Vancouver, B.C., among others.
So far the program can recognize motor vehicles accurately more than 95 percent of the time, but struggles more with pedestrians. This is because of the “boxy” nature of motor vehicles and their more predictable travel patterns. A computer has a harder time trying to understand the differences in pedestrians due to the variety of forms.
“The pixel pattern [for vehicles] is easily identifiable,” said Loewenherz. However, the “nuance between a person in a wheelchair or person on a bicycle is much harder.”
While Bellevue continues to serve as the “test-case,” Loewenherz said he hopes this program will eventually get to a place where multiple cities can access the tool, understand where problematic corridors are and proactively fix them. The ultimate vision, he said, “would take the form of a trusted data collaborative.”
“This would be a nonprofit organization," he added. "Early thoughts are that the University of Washington would take on this role, they would serve as the trustee. They would onboard municipalities that are interested in generating the kinds of data that we’re talking about. … Each city would then have their own unique data vault.”
While this model would not only allow municipalities to understand their own traffic system, the data could also be mined to draw greater insights into transit systems across cities. And cities would be able to keep certain data private if they requested.
The project will launch a beta dashboard for its partners in July. The program is already “able to take raw video footage, translate that to data points for turning movements at intersections,” said Loewenherz. And by winter of 2017, a full beta that can predict more than motor vehicles is expected to be released.
Loewenherz explained that this work melds well with the work in the autonomous vehicle space, but goes beyond that. This is a public health crisis. As the ninth leading cause of death, worldwide, reducing traffic fatalities needs to be tackled with everything possible.
“We need to get more ways to get people interested in this," he said, "and not have it limited to transportation geeks like myself.”