IT Metrics Help Determine Performance of New IT Deployments

Measuring the value of public-sector technology.

by / February 11, 2011

Budget crunches are everywhere in state and local government. Recently elected leaders and officials carry promises of fiscal restraint with them as they face their constituents.  

Many are taking bold initiatives at the outset of their terms to show they mean business. For example, newly elected New York Gov. Andrew Cuomo volunteered to cut his pay before cutting state workers’ salaries, and he has threatened to shut down the state if a budget isn’t passed. California Gov. Jerry Brown plans to cut $12.5 billion in a state that has faced and will continue to face dire fiscal challenges.  

Amid this landscape of belt-tightening is a fiscal topic that most governments must face: IT spending. Policymakers — not to mention taxpayers — want to see a return on any IT spending; tools, and technologies must save money, improve service and boost productivity with a reasonable payback. If governments can demonstrate this to multiple stakeholders that range from ordinary citizens to state assemblies and city councils, then IT spending can be spared aggressive cuts.  

To justify and substantiate IT investments, government leaders and managers must sharpen their ability to communicate value that constituents can understand and embrace. If analytics are available, it makes the sell that much easier. Larry A. Godwin, director of the Memphis, Tenn., Police Department (MPD), is not a CIO, but he knows the value and importance of IT as a public good.  

“As an organization whose mission is to create and maintain public safety — with focused attention on preventing and reducing crime, enforcing the law and apprehending criminals — the Memphis Police Department is held accountable on a daily basis by the citizens it serves,” he said.

The department uses analytic software to  guide strategic planning and day-to-day activities. The analytic tools can evaluate crime patterns in areas as wide as the entire city or as narrow as a single block, allowing the MPD to find crime hot spots and deploy additional patrols.

Godwin said the software produces results that are difficult for any steward of the city’s budget to argue with. Predictive analytics is part of the MPD’s Crime Reduction Utilizing Statistical History (Blue CRUSH) program, which cut serious crime in Memphis by more than 30 percent, including a 15 percent reduction in violent crimes.

Godwin said this is only the beginning of the payback as the increased use of technology in crime-fighting activities has improved communications with other levels of law enforcement and government. “Through Blue CRUSH, MPD works hand-in-hand with the district attorney’s and U.S. attorney’s offices to ensure that Blue CRUSH defendants are prosecuted in the most appropriate venue,” he said. “Today Blue CRUSH has expanded tremendously and works in tandem with MPD’s Real Time Crime Center [RTCC], a $3 million state-of-the-art crime monitoring and analysis hub that opened in April 2008.”

IT Metrics: A Necessary Evil?

What the MPD did, however, may not be possible for all governments as it requires additional budget and effort. Some believe that it may not even be necessary.
Patrick Gray is president of the Prevoyance Group in Fort Mill, S.C., and author of Breakthrough IT: Supercharging Organizational Value Through Technology. Gray thinks that the proof is in the pudding. If taxpayers can easily see the benefit of IT spending, a sophisticated ROI calculation may not be needed.   

“While I’m not sure that the public at large is chomping at the bit for published IT metrics, there is obviously a desire for knowing that the government is reducing costs and spending taxpayer money wisely,” Gray said. “If these types of metrics can be consolidated across government functions, it makes a compelling case for these being monies well spent. I don’t think there’s anything necessarily wrong with ROI measurement, save for when you have to make too many leaps to tie the return to the IT investment.”

Often a challenging task of determining the economic return on a public good is defining who the stakeholders of such public goods truly are.

“The first step is to parse what you mean by the ‘public.’ If you want to be serious about measurement, you have to dismiss the idea that there is a monolithic public, but rather a diverse mix of stakeholders who have different interests and therefore different value propositions,” said Theresa Pardo, director of the Center for Technology in Government at the State University of New York, Albany. “Any measurement model must therefore have a way of representing these interests and how they are affected by the IT project development and operation. The model requires identification of stakeholder groups, their specific interests in relationship to the project’s characteristics, and methods to detect how those interests are likely to be affected.”

Getting public buy-in for government programs can make the justification of an IT investment easier.  

In Florida, Volusia County helped sponsor community forums for a variety of issues for its member agencies. The city of Daytona Beach Shores used a community forum to involve the public in the direction and policy of the city’s strategic plan, explained Phara McLachlan, CEO of IT consulting firm Animus Solutions.

“One of the issues discussed was the creation of a full-time emergency management program in the wake of Hurricane Andrew,” McLachlan said. Citizens rated the program a top priority, even though it required the city to make a significant technology investment, including standing up a new emergency operations center.

“The citizens made this their No. 1 directive because of the intangible insurance benefits to the community,” she said.

While private companies often can evaluate the effectiveness of new technology based on cost reduction or revenue gains, the benefits of government IT systems can be harder to quantify.  

“Traditional ROI measures fail to account for some of the unique functions that government performs, like public safety — for example, what is the ROI of preventing crime? — which contain both hard and soft cost savings,” said Terri Jones, government industry manager of Hyland Software in Westlake, Ohio, who has implemented IT projects for the Pennsylvania Treasury Department. “Government entities exist to provide what economics has traditionally classified as public goods. These goods are difficult to assign costs and value to, and processes like those of the justice system are based in constitutional and legal rights. If they are ignored, there are lawsuits and legal costs that might be costs that could be observed, but it is difficult to assign accurate costs for concepts like due process.”

Policymakers often have mandated that agencies develop and report performance measures, but these initiatives typically lack additional funding for agencies to implement them, Jones said, resulting in ineffective and inconsistent results.

“This has resulted in a lack of baseline cost estimates, failure to consistently monitor and tally savings, and often changes in policy choices might change metrics before IT projects are fully implemented,” she said. “Traditional customer satisfaction surveys can be swayed by political discontent or other factors that may not be directly related to the performance and effect of a technology project.”

In this vein, could it be that traditional ROI models used in the private sector just doesn’t apply? Gopal Khanna, former CIO of Minnesota and the U.S. Peace Corps, thinks so.  

“When it comes to fully measuring the value of public-sector IT projects, the traditional ROI model does not suffice,” Khanna said. “It’s based upon the notion that capital investments in IT projects should result in actual savings over the project’s duration. This is great in theory, but doesn’t accurately measure a project’s value. Governments, like the private sector, must move on to determine other outcomes they are looking for.”

Khanna knows well that in the last decade, governments are following suit of their private-sector peers in ensuring that IT investments achieve savings and gain a competitive advantage in the global market.

“Government too needs to rethink investment philosophy when it comes to IT,” he said.

“As governments struggle to balance revenue with legal obligation, they must look for savings, while working to make services more citizen-centric and providing the productivity gains necessary to offset a dwindling work force.”

In a 2006 white paper, Advancing Return on Investment Analysis for Government IT, Pardo said experts, consultants and local leaders mostly agree that it’s difficult to pinpoint IT ROI in the public sector. Framing the value to citizens and stakeholders is key. The conclusion that she and her colleagues reached then still stands: “The value of a government’s investment in IT should be assessed from the point of view of the public it serves.” ¨

Jim Romeo is a writer based in Chesapeake, Va. He is the author of two books and more than 700 magazine articles. His focus is business and technology topics.


Jim Romeo Contributing Writer

Jim Romeo is a writer based in Chesapeake, Va. He is the author of two books and more than 700 magazine articles. His focus is business and technology topics.