IE 11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

2012 Year in Review: Big Data

The way big data has shaped the way we work and live in 2012.

Editor's note: Over the course of a year, we write hundreds of stories covering myriad topics. And at year's end, we attempt to make sense of it all. We looked for trends that had a profound impact this year — and that are likely to be even more influential in the future. We think our choices fit that mold.

In this Web series, we'll look at social media, which is steadily reshaping how agencies deal with the public; big data, which holds new promise for improving government performance; BYOD, cloud computing and software as a service, all of which are challenging long-held assumptions for how agencies acquire and use technology; and the emergence of chief innovation officers, which hints at eventual challenges to traditional organizational structures themselves. We expect these trends, which took root in 2012, to impact our work and world as we move into next year and beyond.

Year in Review Timeline ... continued


Big Data: It’s Really Happening

From GIS to predictive policing, data streams are being used increasingly to provide better services to communities. Government and law enforcement agencies ramped up their use of big data this year in a number of different ways.

One of the more interesting developments was the first major project of the G7 — short for Group of Seven — an informal collaboration between big city CIOs. The group launched on Aug. 1. The website adds a new layer to the federal government’s open data portal,, featuring data sets from Chicago, New York, San Francisco and Seattle.

While not every data set from each city is online yet, G7 member and San Francisco CIO Jon Walton said his city has about 90 percent of its data on the shared portal and plans to add new data sets to it as a regular practice going forward. The other cities involved also are expected to contribute new data, too. Perhaps even more important, the data is standardized. By making the flood of data available, cities could spur innovation among developers through hackathons and other collaborations to create apps and other advancements to improve the quality of life of residents.

Creating those apps on standardized data sets increases the potential for multi-city development events and more convenient apps for users. For example, the same parking app people use at home in San Francisco could also work for them while vacationing in New York. The approach also makes it easier for G7 cities to evaluate their performance against their peers.

The concept of big data — analyzing large data sets and using the information to make decisions — continues to grow as agencies get better at capturing and sharing statistics on what they do. 

For example, Colorado is taking a look at data from the state departments of education, higher education and a number of others to get a better understanding of why some students prosper and others don’t once they finish school. The idea is to find out whether factors such as early childhood education or a having an incarcerated parent make a tangible impact on a student’s success.

The problem with big data, however, is that much of the information has errors that affect the quality of studies being done from it. To overcome this, Colorado is relying on technology known as master data management (MDM) which analyzes bits of wrong information such as name misspellings and slightly incorrect addresses to determine what data belongs with a particular individual. It cleans up “dirty data” and makes it more reliable.

Another growth area for analytics is law enforcement. Police departments throughout the United States continue to harness the power of data analytics to get a better handle on crime hotspots and in some cases, stop illegal activity before it happens.

For example, predictive policing was a huge success in Santa Cruz, Calif., this year. Through the use of advanced analytics and predictive technology, the Santa Cruz Police Department reported a 19 percent reduction in property theft in the first six months of 2012 compared to the same time period in 2011. No changes were made in the department during that time, except for the use of data-driven prediction methods.

Looking ahead, some experts believe big data will expand beyond statistics into behavioral practices, leading to further success in anticipating crime through technology.

Dawn Clausius, police intelligence analyst with the Olathe, Kan., Police Department, told Government Technology earlier this year that offender behavior and other non-traditional information such as virtual gaming could potentially increase the effectiveness of predictive technology in the years to come.

“I don’t think law enforcement and public safety has even tapped into that as far as a data source or intelligence,” Clausius said. “There are all kinds of games for all different purposes … and maybe on a federal level they are already gaming and in those worlds, but from a local law enforcement level, we are not in any of that.”

Read more 2012 Year in Review stories