IE 11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

How to Successfully Harvest Value from Open Data

Experts offer advice on how civic tech firms, entrepreneurs and nonprofits — who are are pushing state and local governments to open up more datasets based on national standards — can best use the data they seek.

How do governments harvest the greatest value from open data?

That’s the predominant question posed by the civic tech community as officials, entrepreneurs and nonprofits pioneer the movement’s next chapter. Those familiar with open data know what started as an initiative for transparency has quickly evolved into a quest to leverage apps and analytics to enhance decision-making, services and citizen engagement. Success stories on these fronts are many. Civic hackers have created apps to plan bus routes, boost participation in politics, map emergency resources and even to track school buses for parents.

Prevailing logic asserts if leaders publish the “right” datasets — and enough — citizens will find them, civic hackers will use them and the community will benefit with apps and efficiencies. Now, however, this mindset is undergoing a seismic shift as demands for greater gains calls for new strategies.

Beyond the Open Data Portal

Leading the way is Diego May, co-founder and CEO of Junar, the open data portal company. The time for open data 2.0 is here, and for most jurisdictions it often starts shortly after a portal’s launch, according to May. Data must be kept up to date and leadership must invest time to find new outlets for innovation instead of the traditional hackathon. May believes the value from open data comes from private-public partnerships, collaborations with tech startups and companies that link government mission statements to civic tech.

Junar: Top 10 Most Popular Datasets (all jurisdictions)

  1. Budget and general financial data
  2. Mapping and GIS data (transportation, businesses, restaurants, parking)
  3. Building permits
  4. Special projects
  5. Transit
  6. Non-emergency/311 service data
  7. Parks, recreation and tourism
  8. City infrastructure
  9. Parking
  10. Environment and sustainability
Other popular data: Healthcare, education, law enforcement and emergency preparedness
“What we see today is that the real innovation is not necessarily coming from hackathons, but now it’s about working with companies or entrepreneurs to solve problems,” he said.

Because Junar operates in North and South America, May said there are lessons both continents can learn from the other. Bahia Blanca, a city on Argentina’s eastern coast, is one example. Like many South American cities — and medium to smaller U.S. cities — Bahia Blanca lacked a robust developer community. It’s a discouraging gap that can hinder the best leadership from investing in open data when they don’t see tangible results apart from transparency, according to May.

“When thinking about biggest impacts, it’s clear that open data sites, per se, are not the place where most people will end up going,” May said. “It’s more the APIs that we provide — Junar, Socrata, etc., — that deliver this open data to citizens through apps.”

Bahia Blanca’s CIO Esteban Mirofsky grasped the concept. He couldn’t make the city’s innovation efforts wholly dependent on citizens. A new strategy had to be created. To start the process, Mirofsky — along with city leadership — decided to tie open data to procurement initiatives. Next, the city established an office of open government, set an open data policy and launched a portal. Mirofsky then led a successful campaign to partner with entrepreneurs, technologists and vendors to answer civic challenges with open data. A flood of potential solutions came in, and now Bahia Blanca has created a range of civic apps from e-parking and live streaming cameras for industrial monitoring to tracking petroleum refineries and their impacts on the local estuary.

“They didn’t have one of these super-vibrant hacker communities, so they needed to kick-start that themselves,” May said.

Identifying Next-Gen Datasets

There are many ways to quantify the next wave of open data: what they’ll be, how they’ll be used and the various sources supplying them. However, Safouen Rabah, Socrata’s vice president of products and services, envisions next-gen open data selection to be a highly singular process for every jurisdiction.

“Our definition of a second-generation initiative starts with the end goal: What outcomes are we driving?” Rabah said.

If cities and states wish to move beyond mere transparency, then a bit of strategic introspection is in order. Leadership has to ask how data influences quality of life, business, tourism, performance, decision-making, efficiency and other civic issues, according to Rabah.

“The most important mind shift that government leaders can make is to stop thinking about open data as a project they do on the side and start thinking about how they can use data strategically to solve the problems they — and their constituents — care about,” he said.

For this to happen, city leaders need to stop thinking of data as a byproduct of services, and instead view it as a public utility or a commodity that drives government services. This might mean ensuring every city or department data source is fitted with its own API to channel their select datasets, just like a diverting valve might redirect water. This, Rabah said, applies both to private internal data and public data.

Socrata: Popular Datasets by City, County, State

Note: Within each list open dataset categories are not arranged in any order.

Top City Datasets

  • Crime
  • 311 service requests
  • Permits and licenses
  • Businesses
  • Police and fire incidents
  • City payroll
  • Emergency shelter and evacuation
  • Parks and public facilities
  • Adopted budget expenditures
  • Traffic collisions
Top County Datasets

  • Food inspection data
  • Crime data
  • 311 requests
  • Budget and expenditures
  • Election data
  • Foreclosures
  • Census
  • Contracts
  • County services
  • Residential and commercial permits
Top State Datasets

  • Unemployment and employment statistics
  • Business registrations and licenses
  • Expenditures and spending
  • Traffic volume
  • Campaign finances
  • Healthcare facilities
  • Farmer’s markets
  • Weather warning systems
  • Signed bills or laws
  • Air quality

Adding to the advice, May presented a few top trends he’s noticed as well.

“When you talk to cities that are 'state of the art' — or cities that really want to engage in innovation — our answer and their proposition is to bring in real-time data,” May said.

Real-time data apps, equipped with an API, can push data immediately to users, compared to the slower, less dynamic approaches used by most governments. Past examples are parking, permitting and performance analytics. Another value driver: open data that can be mapped and visualized. Mapping and GIS data, according to May, can provide a visual narrative to interpret open data and make it more desirable.

A third and final trend are the governments that apply open data when attempting special projects for public engagement. The proposal for a new park, facility, tax or regulation could be coupled with open data for citizen engagement. “When agencies and municipalities engage in special projects, those are great opportunities to allow open data to increase participation and innovation,” May said.

Adopt a Standard

As with automobiles, appliances or home electronics, the practice of standardization is the next game changer in open data. It’s what open data advocates are buzzing about because it could catalyze innovation across governments. Civic entrepreneurs need apps that touch many markets for viability. If open data standards were adopted, they’d act as infrastructure for a single app to serve users at regional, state or even national levels.

At Accela, technical evangelist Mark Headd is trumpeting the civic tech company’s recent campaign to create the Building and Land Development Standard (BLDS). Targeting the permitting application process, BLDS represents one of the first national standards for open data, and Headd believes it will foster a host of new solutions through apps and analytics. Likewise, the standard undergirds the Accela Civic Platform’s building permit services that, geographically speaking, touch 25 percent of the U.S. population.

“Again, because governments use our core software to manage the actual business of government — instead of just for standing up a platform to publish data — we’re in a position to enable our customers to make richer data available,” Headd said. “As this new standard gets rolled out, we expect to see even more demand for building permit data.”

Nonprofits like the Sunlight Foundation are also pushing for data standards and have released general guidelines to assist policy makers and those like the international Open Data Institute that certify open data through a rubric of quality measures.

Perhaps one of the most ambitious initiatives to standardize data is coming from the federal level. Since President Obama signed the bipartisan Digital Accountability and Transparency Act (DATA Act) in May 2014, the U.S. Treasury and White House Office of Management and Budget have worked to solidify the law’s hold on agencies. The act mandates that by 2018 all federal agencies publish expenditures as open data on, the nation’s searchable and publicly accessible website for finance. As a necessary precursor, the OMB and Treasury are partnering with Socrata to create standards to enable the transition.

Jason Shueh is a former staff writer for Government Technology magazine.