A city’s data is one of its most valuable assets. Urban data is the bedrock of the performance management programs that allow cities to ensure continuous improvement. Reliable data can facilitate interagency collaboration, improve partnerships with the private sector, and expand public engagement. Innovative uses of data allow cities to enforce regulation and improve social services. And, increasingly, open data is serving as the foundation for good government activism, allowing journalists and civic hackers to highlight government inefficiencies or even spot corruption.
Yet these digital resources are often taken for granted. Outside of the dedicated world of civic technologists, many of us imagine that city data is readily accessible and easy to use. But the process of collecting, cleaning, integrating, and analyzing data requires extensive capital investment, interagency collaboration, and long-range vision. In the face of complex organizational and technical challenges, cities are developing strategic plans to guide the development of more open, data-driven city government.
As data assumes a more central role in local governance, cities are taking diverse approaches to planning and strategy. These plans move beyond the kind of roadmaps that have long been used by information technology agencies to set out operational and infrastructure improvements. Instead, they embrace the strategic, technical, and institutional requirements of a rich, cross-agency data environment. Approaches range from San Francisco’s Data in San Francisco, a standalone proposal for open data; to Hong Kong’s Smarter Hong Kong, Smarter Living, centered on information and communication technology infrastructure; and to the Chicago Tech Plan, which positions data as a step towards expanding the digital economy.
With so many ways to plan the city’s digital future, where should a planning process begin? Below, we profile the digital strategies of three US cities, highlighting strengths, weaknesses, and innovations. While the plans vary substantially, they fall into two distinct types: plans that focus on the city’s open data program and plans that address data as part of a comprehensive technology strategy.
Planning for a smarter city government will be different in each place: cities will want to build on synergies with regional partners in the public and private sector, as well as state and national governments, and the residents of each city will have unique concerns. Despite these particularities, data strategy best practices can help guide planning across cases.
As open data programs have grown, many cities are finding that these efforts are a natural site for planning expanded data initiatives.
With “Data in San Francisco,” the city’s DataSF team sets a high bar for open data planning. The strategy is distinctive for its ambitious goals, detailed milestones, and accountability tools to track the team’s progress.
In 2009, then-Mayor Gavin Newsom issued an Open Data Executive Directive requiring city departments to make all non-confidential datasets available to the public. The Directive led to the creation of DataSF under Mayor Ed Lee, an office responsible for publishing these newly-public datasets. But today DataSF does much more, building capacity for data analysis within and beyond government and identifying opportunities for data integration and collaboration.
DataSF published its first strategy in 2014, setting out goals to develop a comprehensive data inventory, relaunch the open data portal as a more user-friendly resource, and cultivate partnerships to enhance the accessibility and utility of open data. Having achieved these goals, DataSF developed a follow-up plan, published in July 2015, which built on this foundation and set out the second phase of open data strategy for San Francisco.
The current Strategic Plan identifies key goals over a two-year timeline. These include continued efforts to increase the number of datasets on the open data portal and improve the timeliness of data publication. As part of this process, DataSF asks departments to submit annual publishing plans for the coming year. These plans are openly accessible via DataSF.org, allowing departments as well as San Francisco residents to track progress. For the departments, these plans help prioritize datasets for publication and provide a regular feedback mechanism between the department and DataSF.
But the strategy also moves beyond these core tasks. DataSF plans to expand the number of private data sources in the portal while enhancing privacy protections. They intend to streamline internal access, so that city departments can more easily find and use data from other agencies. And they aim to continue improving the consistency and reliability of both data and metadata, which is critical to usability. As of February 2017, San Francisco had published 420 datasets and inventoried a total of 860.
DataSF aims to increase not only the supply of data, but also the demand. This is accomplished by fostering a culture that puts data to work for city decision-making and by building capacity within government agencies. Central to this effort is the Data Academy, a program that provides training for city employees in data analysis, visualization, and other skills. The Academy, jointly run with the City Controller, offers a rich array of courses, from introductions to Excel and R to courses on data usability and business process analysis. To date, courses have attracted over 1,000 enrollees and the Data Academy plans to train 600 this fiscal year.
Perhaps most impressive, DataSF tracks its progress with fine-grained milestones in publicly available Google documents, updated quarterly. They prioritize goals according to the “must, should, could” (MoSCow) method, to ensure that the most pressing tasks are the first to be completed. And while the reporting document serves as a highly transparent tool for public accountability, it is also designed to be useful to employees within DataSF. Internally, quarterly goals are broken into monthly goals, assigned to team members, reviewed at monthly meetings and tracked with regular progress reports. Far from being a public relations tool, this reporting method actively guides workflow and ensures that activities across the organization are coordinated and mutually supportive.
New York City has been a leader in many ways: the 2011 Digital Roadmap was among the first technology strategies published by any city. And yet, when it comes to planning for data, the city sticks to a practical strategy focused on open data, with annual updates to the open data plan that identify which datasets are set to be released and when.
The 2015 report, Open Data for All, established a stronger focus on the needs of citizens and communities, aligning with the priorities of the De Blasio administration. It follows on two previous annual plans, required by the city’s open data law and first implemented in the final year of the Bloomberg administration. Despite the lofty title, the strategy is simple, articulating five new goals: to talk and meet with citizens in order to understand their priorities; to shift from annual to monthly open data plan updates; to release new datasets; to structure data publication around end-user value; and to improve infrastructure and increase responsiveness.
Following an overview of these aims, the plan lays out the datasets that the city intends to publish in the near term. The plan identifies 90 new datasets to be published by the end of 2015, and 250 additional datasets by 2018 — when the city is required to have all data publicly available, per the legislation that put open data into effect. By February 2017, 1,600 datasets had been published.
After De Blasio’s administration came under fire for missed deadlines and allowing agencies to set their own schedules for releasing new datasets, City Council passed amendments to enforce compliance with the law. The Mayor’s Office of Data Analytics (MODA) has stepped in to help ensure that datasets are identified and published in timely matter, embarking on “friendly investigations” of agencies to ensure that they meet deadlines. In December, they released the first examinations and verifications report about the open data compliance of the Department of Sanitation, the Department of Correction, and the Department of Housing Preservation and Development, finding all three in good standing. This formal review process for agency participation in a data plan enables the city to quantify outcomes and measure its success over time.
Beyond open data publishing, MODA plays a central role in cultivating a culture of analytics by leading innovative cross-agency projects. But the responsibility for data initiatives is spread across multiple agencies, including MODA, the Department of Information Technology & Telecommunications (DoITT), and the office of the Chief Technology Officer (CTO). Large agencies with major data needs, such as the New York Police Department, also collect and manage their own data resources. To date, the city has not developed a formal process for interagency data planning.
In contrast with plans rooted in open data programs, some cities envision the future of urban data in the context of community, economic development, and infrastructure goals. While this approach is common among international cities from Dublin to Singapore, the most prominent US example is found in Chicago.
Mayor Rahm Emanuel saw what data and analytics could do for the White House as President Obama’s chief of staff and brought this experience to Chicago. Upon taking office in 2011, he set about putting technology at the forefront of Chicago’s plans. This commitment was summarized in the city’s lengthy, wide-ranging Tech Plan, published in 2013. The stated goal was to “[r]ealize Chicago’s potential as a city where technology fuels opportunity, inclusion, engagement, and innovation for all.” The plan aimed to encourage businesses and community groups to join forces with the city in pursuit of technology goals.
The Chicago plan encompassed a diverse set of goals connected to the quickly-evolving world of technology. The city identified five main strategies, with specific initiatives for each one.
- Infrastructure: Chicago focused primarily on enhancing broadband infrastructure and expanding access, although the strategy also identified technology-based improvements to the transportation system.
- Community inclusion: The city, which suffers from high rates of segregation and inequality, dedicated nine of its 28 initiatives to improving community access to digital tools and capacity to take advantage of technological resources. Initiatives included free public Wi-Fi, increasing options for low-cost broadband, expanding digital training, and supporting existing community programs.
- Efficient government: With the aim of improving the city’s efficacy, this strategy focused on the collection of data, the improvement of open data resources, and the use of data to drive city decision-making. In addition to data strategy, this component included a shift toward enterprise resources within government, including consolidated data centers and the promotion of enterprise software solutions.
- Collaboration: Within this strategy, the city aimed to expand collaboration with the civic tech community to address large and small city problems. Many of these collaborative initiatives depended on the strength of the city’s open data program.
- Tech sector: Finally, Chicago set forth an ambitious agenda to support the growth of the tech sector, including the direct provision of incubator and co-working spaces, programs to connect startups with resources, and pipeline development through the cultivation of a local STEM workforce.
For each of these strategies, the city set out measures of impact in terms of savings, services, engagement, access, skills, jobs, and STEM education. An 18-month update was published in 2015, reporting progress on each initiative. In the report, the city highlighted its accomplishments, including a tenfold increase in IT infrastructure within schools, expanded public Wi-Fi offerings, one-on-one digital training for 100,000 citizens per year, and new online tools for licensing and permitting. The update, while providing useful information on the city’s accomplishments, did not provide the kind of rigorous accountability that specific milestones and more frequent reporting makes possible.
In the context of such a broad plan, however, Chicago gave data only a partial role. As the capacity for big data collection and analysis grows, it is possible to imagine a plan that puts data at the center, uniting otherwise disparate initiatives through the prism of data integration, transparency and accountability, and the potential for economic growth through innovation. The role of Chief Data Officer, central to developing such a comprehensive, data-driven vision, has moved from the Mayor’s Office to the Department of Innovation and Technology. This move made the CDO simultaneously autonomous from political priorities and deeply embedded in the operational workflow. From this position, the CDO can work directly with agencies across the city and has a much stronger chance of continuing as a robust office into future administrations.
As these plans illustrate, there are many different ways to plan strategically for the future of data. Cities looking to develop their own data strategy will want to consider the benefits and drawbacks of different models.
- Guiding vision, general goals, or concrete milestones? A guiding vision can be inspirational, and concrete milestones can seem too dry and in-the-weeds to generate excitement within and beyond government. The middle ground — general goals — has the benefit of flexibility, with different applications in different departments or sectors. But without concrete milestones, it can be difficult for the public to understand how much (or little) progress government is making. More importantly, it can be difficult for the government to accurately gauge its own successes and identify pain points.
- Is the audience inside or outside government? Politicians want to share their vision for the future with the public, but ultimately the plans to achieve this vision must be implemented by people within government. Public-facing strategies can be inspiring even for government workers, but should be accompanied by more detailed internal documents to concretely guide action across diverse departments.
- Who informs the plan: internal or external stakeholders? Ideally, a strategy would be informed by the experience of people within government as well as by the constituents that they serve. The fact is, however, that resources are limited. In developing a plan, cities should consider where they have a greater gap in their knowledge. Do we know what our citizens want? Do we know what problems are facing our departments? The smartest planners will get the input of those who can fill these gaps.
- Single agency or inter-departmental implementation? The achievement of broad plans requires the participation of many departments, and collaboration can often be difficult. Single-agency plans are easier to implement, but narrower in scope.
- Authorship and oversight? Leadership is key to accomplishing goals in city government, especially when those goals span multiple agencies and require substantial human and capital resources. A plan presented by the mayor and implemented by City Hall may thus be able to achieve goals more quickly. At the same time, City Hall might have limited resources to dedicate to data projects, and is subject to transformation from administration to administration. A focused team within an operational agency, as in Chicago, or as a freestanding group might allow for more continuous attention.
- Integrated or separate plans for infrastructure and data? Data has a critical role to play across all technology initiatives; similarly, infrastructure is the foundation on which data is built. By embracing both data and infrastructure, cities can ensure that these two areas develop in tandem, but trying to tackle both at once may tax limited planning and implementation resource.
- Timeline? Many great projects require long timelines, but city government operates according to election cycles. To ensure that the data plan reaches its goals, effective strategies often focus on the near term, in two- or three-year increments.
- Know your data holdings: To have an effective plan for publishing data, you must first know what data exists to be published. The first step in a comprehensive data plan is thus an inventory of all datasets held by the government. The Sunlight Foundation advocates publishing this inventory as a first step toward government accountability.
- Create data standards: As Stephen Goldsmith highlights, data can be understood in multiple ways. In order for open data to facilitate better collaboration within government and beyond, cities must establish clear and consistent data standards.
- Implement data governance: Clear governance is essential to protecting the privacy of individuals and the security of sensitive data.
- Governments can move beyond city data by investing in data infrastructure: There is lots of useful urban data that doesn’t originate in the city government. One of the most effective things a government can do is to create widely-useable open data infrastructure that can host public sector, private sector, and even individually-generated data.
- Before you have a data plan, you need an open data policy. The Sunlight Foundation identifies best practices for open data policies, including proactive publication of datasets, transparency regarding government data holdings, established processes for identifying priority data, and public APIs for accessing information.