Assessing Your City’s Data-Driven Approach

This story was originally published by Data-Smart City Solutions.

This self-assessment tool can help city leaders determine the degree to which they are using data to drive better results for the public.

Cities are increasingly paying attention to data

Increasingly, cities are using data to better understand and serve their communities. Many mayors have appointed data experts to new positions to lead this effort, and while some cities have adopted the Chief Data Officer role, others have appointed data-savvy Innovation Directors or Performance Officers. Regardless of the title, the focus on data in cities is growing.

In the leading cities, advances are accelerating as they build on each others’ momentum

The group of Chief Data Officers who constitute the Civic Analytics Network (CAN), a peer network of data leaders from U.S. cities, are leaders who each contribute in distinct ways to the data-driven government conversation – some bring data analytics acumen, others bring technical infrastructure and data governance expertise, while others have experience in geographic information systems, predictive analytics, or organizational culture transformation. What they all have in common is that they are essential leaders in their respective cities, sought out to bring new approaches to the most vexing city challenges. They’re solving important problems using data visualization and sophisticated analytics tools to find more efficient ways to keep streets clean, make restaurant food safer, reduce lead paint exposure for children, combat rodents, increase the fairness of real estate taxes, improve traffic flow, and curb opioid addiction and homelessness. They’re data champions in their cities, offering new approaches like human-centered design and behavioral insights and nudges to help improve city life.

The volume, diversity, and sophistication of data efforts in the leading cities is growing

The rapid expansion of city-level data-driven decision making is apparent in the growing sophistication of the projects and capabilities of CAN cities over the past few years. When CAN began, only Chicago and New York City were engaged in predictive analytics. Today, the majority of network members either have a data scientist on staff or have engaged outside resources for advanced data services such as predictive analytics. Since the network launched two years ago, network conversations have evolved, too. Initial discussions focused on the basic blocking and tackling of a startup enterprise–staffing models and procurement challenges. Now, CDOs talk about emerging issues like technology-driven mobility, sensors and other smart city strategies, combatting algorithmic bias, and integrating city data with other sources. The CAN cities have individually and collectively advanced along a continuum of greater use of data to provide value to the public.

How can other cities assess where they stand in relation to the leaders?

What can a city do to see how it measures up to these leading examples of data-driven government from the Civic Analytics Network? The framework below provides the foundation for a self-assessment tool to help cities determine where they are now and what they can do in the future to become more data-driven. This framework follows a four-stage capability maturity model (see below) and was inspired by observations of the advances made by members of the Civic Analytics Network. For more detail on the maturity model see “Analytics Excellence Roadmap.”

What is common at the highest level of data-driven government is strong leadership that establishes the culture–a culture of valuing the use of data to set priorities and allocate resources via a variety of tools and methods ranging from performance measurement, to data visualization and geospatial maps, to advanced analytics.

The framework describes a generalized path showing that as cities mature in their capability to produce, share, and use high quality data, they provide the opportunity for both internal and external users to analyze and use data, which in turn enables improved government performance. This framework defines open data as a foundation and an accelerant of data-driven government. While not a necessary precursor to creating a data culture, the process of publishing open data provides ample opportunity to assess data availability and quality and allows cities to become more familiar with data previously unavailable or unknown because it was siloed in a different department or hidden in a file drawer. Opening and sharing data facilitates its use for analytics, management, and resource allocation.

The self-assessment tool

For each stage of the maturity model, there are a series of questions that can help a chief executive or data leader assess the degree to which their city government is already prepared to share and use data, and the areas in which they have room to grow. The questions in the self-assessment tool provided below are intended as a starting point for discussion and offer a way to help a leader interested in moving toward greater use of data get started–this is not a one-off self-diagnostic tool, but rather should initiate an ongoing discussion of how to continuously become more data-driven.

How to use the tool

Chief executives and data leaders alike can use these questions to guide their thinking as they develop strategies for becoming increasingly data-driven. For more on the key elements of a good data strategy, see, “Lessons from Leading CDOs.” Revisiting the questions below annually will help provide a way to measure progress toward achieving strategic goals. Today, no city, not even the top leaders in the field, can answer every question below in the affirmative. All cities can and should aspire to continually improve the degree to which they provide the public with transparent operations and services, hire and train analysts with the skills and tools to use data, and equip managers and leaders with the competency and confidence to make decisions based on data-driven insights.

Since the purpose of this tool is to help cities advance in their progress toward data-driven government, we’d like to hear from you about your experience using the tool. Send us your feedback at datasmart@ash.harvard.edu.

Get started

In each of the drop-downs below is a set of questions that can help cities assess their data-driven practices and identify places for improvement.

Publish

Basics:

Does your city have a website?
Is every major city activity, function, or department on your website?

Data governance:

Does each department have someone responsible for updating website content and publishing open data (i.e., a data steward)?
Are publishing formats and standards consistent across departments?

Data culture:

Are tools and training available for data stewards to perform a data inventory and to manage the process of publishing data?

User experience:

Can a new user quickly find what they need without knowing the name of the department?
Is navigation user-friendly and based on customer needs rather than government processes? Is there a common look and feel across the site? Is information shared using plain language?
For the most frequent transactions, can a user perform the function online?

Content:

Are the most frequently requested public documents available online and easy to find? Are statutorily required public disclosures or publications available online?
Is there a tool to automate Freedom of Information Act requests, or a process by which requested items are considered for open data release?

Polish

Strategic vision:

Does your city have a data policy and data strategy? Are those policies publicly available? Do you revisit the strategy regularly (preferably every three years)?

Transparency:

Do you report annually on progress toward the goals set forth in the data strategy?
Is performance data shared publicly for at least some departments?

Data governance:

Are data publishing, analysis, and visualization tools standardized across the city to enable greater cross-training and skill development?
Is open data provided in machine-readable format?
Is open data regularly updated? (for some data annual updates are sufficient, in other cases monthly or quarterly updates are preferable). Is there a process for review and sunsetting for data that is no longer relevant?
Are there internal auditing or other feedback methods that monitor and help to improve the quality and accuracy of open data?
Are there civic engagement feedback loops for public input on data availability or improvements? Are analog and digital tools and social media being used to communicate with and engage the public on city data and operations?

Culture change:

Is training on basic data skills available to city employees, or at a minimum to data stewards?
Is there a data community of practice for peer learning?

Analyze

Culture change — citywide leadership:

Is there a designated citywide leader for data efforts in your city, such as a Chief Data Officer or other data, innovation, performance, or analytics leader?
Does the citywide data leader have the support of the mayor?
Does the citywide data leader have sufficient resources to accomplish her mission (data analysts, data scientists, technology tools and platforms, etc.)?

Culture change — departments:

Is there a designated leader in each department responsible for providing public data, such as a data steward?
Is there a leader in each department for using data to improve operations, i.e., a data champion?
Are there documented successes in applying data analytics to solving the challenges faced by city departments? Are city leaders being recognized for their success in using data for operational improvements?
Is training available to city staff to improve data literacy and support stronger data skills?

Data governance — high quality data:

Is open data provided in a consistent format across agencies?
Is open data well organized and easy for the public and other city staff to navigate? Is data easy to extract and use for research and analysis?

Data governance — analytics platform:

Does the open data portal provide analysis and visualization tools for both city staff and the public?
Do APIs allow for development of user-defined apps across a variety of departments or data types?

Optimize

Data governance — data availability:

Are all city departments providing high quality, operational, and performance data to the public in a timely fashion and in a format that can be easily understood to track progress?
Are all city departments providing real-time updates for key data sources?

Engagement:

Are there written case studies of data analytics successes to document public benefit?
Does public engagement with data produce meaningful feedback on customer viewpoints and satisfaction with operations?

Data use and culture change:

Do department leaders use analytics to support decision-making for all top priority activities?
Do mid-level managers have the skills and confidence to use data to inform decisions?
Does every department have someone with data analytics skills, or enthusiasm for engaging with others on data efforts?
Can city staff share data across departments to collaboratively solve problems, make better policy, and/or deliver better services?
Are skill development opportunities available to data analysts and data scientists?
Is there a network of data leaders across departments for peer exchange and support?

Impact on the public:

Is data being used to help the city to go deeper than basic operational insights to consider issues like equity and fairness of results?

IE 11 Not Supported