State of the Union of Open Data 2016: Despite Steady Gains, Challenges Remain

Looking forward, the goal is for practices associated with open data to become simply a part of good government management in the future, rather than being viewed as a separate concept.

by / November 2, 2016

For advocates of data transparency in government, there is much to feel good about these days. Public officials aren’t just taking an increased interest in open data. They are doing so for increasingly good reasons.

“There is a shift in focus in why governments are transforming their information to open data,” said Hudson Hollister, interim president of the Data Foundation, speaking on the release of The State of the Union of Open Data 2016, a new report co-published by the Data Foundation and Grant Thornton.

“Originally they were doing it because it is a good thing for public transparency, and that remains true,” he said. “But now they are realizing that standardizing and publishing these data is also good for management. It helps internal decision-makers. That was not part of the motivation five years ago.”

The report draws its insights from conversations with more than 40 participants in the recent Data Transparency 2016 conference, held Sept. 28 in Washington, D.C. That event featured three program tracks: Open Data in Management, Open Data in Regulation and the first-ever White House Open Data Innovation Summit. The report includes insights from congressional and agency policymakers, open data experts and advocates, and tech-sector leaders. 

In compiling the report, researchers found reason for optimism. For example, some 90 percent of interviewees see steady gains in open data. They say that the standardization and publication of government data in their fields has improved to at least some extent over the last few years.

But the report also cites challenges facing the open data advocacy community, especially in the area of standards. In the report, participants urged the federal government to end the use of proprietary data standards like the DUNS Number, which is owned by Dun and Bradstreet Inc. and is used to identify federal contractors. They pointed out an urgent need to mandate and develop more universal standards.

Government already is heading down that road. Under the DATA Act, the federal government has set itself the goal of standardizing formats for data related to spending starting in 2017, under the guidance of the Treasury Department and the Office and Management and Budget. This could be the start of a larger movement toward government-wide standards.

“Hopefully the application of standards could be broadened out from here to include other kinds of data, especially performance data,” said Robert Shea, Principal at Grant Thornton. 

“The hope is that we can eventually get standards to apply to performance information, so that we can better see how programs are performing across government,” he said. “Right now you have reporting, but it is not in a consistent format. If you had standardized formats for that, combined with better finance reporting, then you could better compare the cost-effectiveness of programs.”

Many voices

In compiling the report, researchers were struck by the “great diversity of stakeholders” presently engaged in the open data conversation, Hollister said.

  • On the legislative side, for instance, Rep. Seth Moulton, D-Mass., commented on the pressing civic imperative for better transparency. “Investing in civic technology and committing to government transparency are key to a modern democracy," he said. "In 2016, citizens expect and deserve access to information about the laws that impact them on a daily basis."

  • From the agency perspective, USAID Chief Data Officer Brandon Pustejovsky noted that before the community can achieve standardization, it needs to meet certain technical thresholds. “Standardization is extremely important, but we are still at an intermediary step — namely, ensuring the broad availability and machine-readability of our data on a routine basis," he said. "Standardization will become more of a priority as we get these initial steps in place."

  • On the academic side, researchers heard from Beth Blauer, founder of the Johns Hopkins University Center for Government Excellence. She described the need for government to look beyond data for its own sake, to begin addressing data as a fundamental element of better governance. “Government agencies are themselves the prime users of government data,” she said. “Data should be recognized as a strategic asset, and therefore a data strategy connected to real goals and outcomes is critical.”

From Hollister's perspective, this breadth of interested parties bodes well for the open data community. “We see strong support in federal, state and local government officials," he said. "It is there in the private and public sectors, it is the nonprofit community and it is there in the private-sector service providers."

The road ahead

Looking forward, interviewees described a range of steps needed to ensure the continued rise of open data in the public realm.

Some talked about the need to better understand the challenges posed by legacy systems, which can stymie the process of data extraction. They described the need to build in open data processes natively within new programs and systems, when they are first created. In this way the practices associated with open data become simply a part of good government management in the future, rather than being viewed as a separate concept.

Despite the positive feedback, however, the future of open data is far from secure.

With a change in administration due, the possibility of shifting priorities looms large. “There is a big risk that through the transition, we lose momentum,” Shea said. Once DATA Act implementation gets underway, “we will need to keep the momentum going. That will require clear leadership.”

While Treasury and OMB have dominion over spending data, champions will need to emerge to ensure standardization of other forms of government information. “We need to have somebody put in charge of data standards for financial regulation, somebody put in charge of data standards for performance,” Hollister said. “There needs to be standard bearers across government.”

Adam Stone Contributing Writer

A seasoned journalist with 20+ years' experience, Adam Stone covers education, technology, government and the military, along with diverse other topics. His work has appeared in dozens of general and niche publications nationwide.