“‘Are people innately altruistic?’ is the wrong kind of question to ask. People are people, and they respond to incentives.” -- Steven Levitt & Stephen Dubner (SuperFreakonomics)

One of the most important things an open data directive can accomplish – whether it takes the form of an informal policy, an executive order or an open data statute – is to help create a set of incentives that can foster a "data culture" inside government.

Open data has the potential to transform how governments operate by changing the way government officials look at and use data. But the bureaucracy has a number of built-in disincentives that can work against this outcome if they are not addressed through a well-designed open data policy.

Open data is about more than just releasing data to external consumers – though this is a critical outcome that is required if governments hope to achieve additional benefits. It’s about training government officials to think about the power inherent in data.

It’s about creating a data culture in government.

Institutional barriers to open data

When most people think about barriers to open data, they think about the technological and political barriers to governments opening up more data for outside use. Many of the systems that house government data are decades old, and extracting information to release as open data can be time consuming and difficult. In addition, there is still resistance in some corners of governments to releasing information that could be used by outside parties to evaluate the performance of government agencies.

But beyond these more obvious barriers there are other, less obvious impediments that may work against governments both releasing data to outside users and also to making more effective use of data themselves.

Open Data Benefits Mismatch

One of the more common impediments to open data is when a mismatch exists between the potential benefits of opening data to outside users and the responsibility of maintaining it.

In many of the cities that I have talked with, data on the locations of fire hydrants is often thought of as sensitive data. While it is true that hydrants represent an important part of municipal water systems, they are visible to those around their locations and this data is released to the public in some municipalities to support civic applications like Adopt-a-Hydrant.

In some cities, the responsibility for managing data on fire hydrants rests with the Water Department – it is this department that is charged with managing water infrastructure and usually has responsibility for disseminating information to those that need it – other government officials, contractors, regulators – on the location of water infrastructure assets. But the primary beneficiaries of the release of hydrant location data are entities like the Fire Department, Public Safety and Emergency Management.

An open data “benefits mismatch” exists around fire hydrant data because the benefits of releasing the data don’t accrue to the department that is responsible for maintaining it (and that will likely be held accountable if the data is inaccurate). This can create a strong disincentive to releasing such data, and should be taken into account when crafting an open data policy.

Problem Scope > Agency Mission

Another impediment to governments releasing data – and to using it more effectively themselves – can occur when the scope of an issue that governments are charged with addressing is larger than the mission of any one department.

Take for example the issue of vacant property – which is an issue that faces almost every municipal government in the U.S. This is a huge, sprawling, multi-faceted issue that often goes beyond the focused mission of any one agency that is involved with it. The problem of vacant and derelict property touches on crime, poverty, revenue collections, zoning, environmental protection and many others. In cities like Philadelphia there may be half a dozen or more separate departments charged with addressing one or more aspects of this complex issue.

Consider the number of different city departments that may have information on the scope of the vacancy problem. The Department of Licenses & Inspections views the world of vacant property through the lens of enforcement and counts those properties that it has issued violations against. The Water Department views the world of vacants through water service, and counts those parcels that have an active account (and those that don’t). The Revenue Department views the world of vacants through collections, keeping track of those that are current on their property taxes and those that are not (some in arrears many years).

So how many vacant properties are there in a city like Philadelphia? There isn’t a 100% definitive answer because there isn’t a department that is charged with keeping track of vacant properties. Each of these departments has some data on the problem of vacancy, but their picture of the problem is viewed through the lens of the service they provide.

What’s worse, there is no overarching authority that is formally charged with bringing these disparate data sets together and constructing a model for identifying the true picture of vacancy. Each is stuck in a pattern of looking only at their own data. There are no incentives for these agencies to combine data or cross check it against the data of other departments that might have insight into the problem of vacancy.

This is also data that could be enormously beneficial to the public – as vacant properties negatively impact housing values – and other departments, like the Police Department – because vacant properties are thought to be related to the crime.

This is a huge missed opportunity because of the potential analytics goldmine that could be generated if there was some focused effort to combine information on the problem of vacancy and keep track it. Since there isn’t a clear picture of the problem of vacancy, how do we know if we’re making progress in addressing it?

Creating a Data Culture

The problems discussed in this post are not unique to any one city, county or state – the fact is that most governments are not set up structurally in a way that encourages the efficient sharing of data – with the public, or with other government agencies.

This is what makes an open data policy so important.

Good open data policies can offset the structural impediments to open data and create a healthy set of incentives that fosters data sharing both inside and outside government.

mark headd, mark head, philly mark headd Mark Headd is the former Chief Data Officer of Philadelphia. He is now Technical Evangelist for Accela. This story was originally published on Mark Headd's personal blog, reprinted with permission.