Running the Numbers
State ROI programs attempt to quantify the benefits of e-government investments.
When Missouri CIO Gerry Wethington proposed a multi-year electronic government initiative last year, he knew he'd need to show results to keep dollars flowing into the project.
That requirement spawned the Missouri Value Assessment Program, a nascent effort to build a framework for measuring both total cost of ownership (TCO) and return on investment (ROI) for technology projects undertaken by the state. Missouri will flesh out details of the program this year and begin applying it to e-government projects in fiscal 2004, giving the state new insight into the costs and benefits of IT undertakings.
"It will let us begin to make assessments and decisions about funding a project or developing a new service based upon some true data," said Wethington. "That moves you from having emotional debates about projects to having factual discussions."
That's an evolution a growing number of public jurisdictions are looking to make, particularly as they pump more resources into electronic government initiatives. Lawmakers and others who foot the bills for big-ticket e-government undertakings are asking what constituents get in return for that investment. And efforts under way in states such as Missouri, Iowa and Wisconsin are designed to provide some concrete answers.
Many believe long-term support for e-government initiatives will be tied to the strength of these answers, particularly in the current weakening economy.
"[States] need to have some very hard justification in order to sell these initiatives to the legislature and have them funded not just as one-time opportunities, but also to fund the ongoing infrastructures that are needed to continue running them and expanding them as necessary," said Patricia O'Donnell, vice president of marketing for Anexsys, an e-government firm that recently published a guide to ROI measurement for public agencies.
The Right Tools
But turning ROI concepts into workable assessment tools raises some unique hurdles for public agencies, where the results of online programs and services often defy simple dollars-and-cents measures. Although some e-government initiatives -- e-procurement and online vehicle registrations, to name a few -- can deliver hard savings by lowering overhead or transaction costs, many yield intangible benefits such as quality-of-life improvements and better convenience for citizens.
That forces agencies to take an unconventional approach to measuring technology's benefits. Missouri is examining everything from gas prices, to median salaries, to population density in an effort to determine what saving citizens a trip to a government office is worth.
"You've got to look at a whole different set of demographics, and we're still trying to uncover what those demographics are," Wethington said.
He expects to use the state's average salary to put a rough value on citizens' time savings. In addition, Missouri intends to account for travel cost savings by factoring in the distance between government offices and typical users of a particular service.
Wethington acknowledged that figures generated by these techniques would be rough -- but better than nothing. "Most states don't have access to historical data; for the most part you don't have a benchmark," he said. "But in starting to do this, you're building and collecting empirical data for the first time."
The accuracy of that data will improve as governments gain experience with ROI measurement, he added. "You're going to see the benchmark begin to change and adjust and refine itself. This may be something that takes three to five to seven iterations before you finally feel comfortable with how those measures begin to flow out of this."
Anexsys followed a similar path in its guidebook, "Measuring ROI in E-Government." The company developed a series of templates -- available online -- intended to give government organizations a starting point for creating ROI measurement tools. The company teamed with Indiana University and e-government consultant Meta Group to produce the material.
The templates attempt to quantify customer satisfaction improvements and administrative cost savings derived from e-government projects by examining a series of interrelated issues.
"When you are looking at intangibles, it's critical to measure what kind of time savings they represent for the constituency. And if you're going to measure that, you need to be able to assess value to it," said O'Donnell. "The way that we assessed the value was to take the median hourly income by state, which we obtained from the U.S. Census Bureau."
So, if an online driver's license renewal system saves a citizen two hours by eliminating time spent driving and standing in line, and the median salary in that citizen's state is $20 per hour, then the savings per citizen is $40.
But that's only the beginning. Online applications also tend to outsource data-entry chores because citizens type in their own information. And citizens entering their own data often make fewer errors because they have a vested interest in making sure the information is correct, O'Donnell said.
Together, the internal and external benefits form a powerful combination. "You can grow that paltry $40 into a big number if you're a populous state. But if you think about getting at least twice that in return on the back end -- because that's really the hidden cost that people don't think about -- then it's wonderful," she said.
Measuring ROI is closely tied to determining the total cost of ownership for IT systems and equipment. Indeed, Missouri's Value Assessment Program (MoVAP) includes equal parts ROI and TCO.
"It became apparent that you can't even think about the kind of savings you're going to generate until you really understand the cost associated with programs," Wethington said. "So calculating your return on investment means that you need to think in some pretty different terms relative to total cost of ownership of programs within government."
Indeed, MoVAP's impact may send ripples of change far beyond agency IT departments.
"It affects not just information technology and the technologist, it works its way into the whole business process. It works its way into the appropriations process and the budgeting process," Wethington said. "So it's very difficult to say that I'm going to confine this to information technology."
Similarly, ROI measurement is part of a comprehensive IT strategy being developed by Wisconsin's Department of Electronic Government. That strategy, known as the enterprise portfolio management plan, is meant to make the most of new technology investments, reduce the risk of project failure and improve the overall quality of enterprise IT initiatives.
"We're taking a very broad approach. We're in the process of developing an IT enterprise plan," said Shirley-Eckes Meyer, deputy secretary of the Department of Electronic Government. "A component of the plan includes portfolio management, but it also includes factors like changing laws to support the portfolio and how we're going to reengineer business processes."
As part of that effort, Wisconsin will develop tools for assessing both hard and soft benefits generated by e-government projects.
"What we will do is ensure that everybody has perhaps a concrete measure of cost or a concrete measure of service, and maybe customer satisfaction if they are in the business of dealing with the public," said Alison Poe, head of the Department of Electronic Government's Bureau of Strategy Development.
The assessment methods haven't been finalized yet, but Wisconsin already has been gathering results from some IT initiatives for several years, Poe said. For example, it's creating baseline data to show how automated fingerprint identification technology reduces the amount of time it takes to identify criminal suspects.
The state hopes to get a tighter grip on less tangible benefits, as well. "One thing we've done a little bit of in the past, is conducted focus groups with citizens to determine whether they are satisfied with the service they are getting," Poe said. "As we go to more online services, that kind of assessment will probably expand."
Implementing the type of ROI and TCO measurements contemplated by Missouri and Wisconsin likely will demand significant adjustment from government executives. Wethington said agency managers usually aren't accustomed to tracking these issues.
"If you're managing toward the bottom line, that means you've got to do some pretty different things in determining cost and assigning monetary value," he said. "That's a whole different environment; we typically don't think in those terms."
The magnitude of that transformation became evident when Wethington attempted to get MoVAP ready for use in Missouri's 2003 fiscal year, which starts in July. "As we got into the models and the methodology, it became far more complex, and quite honestly this is a somewhat of a culture change for people who deal in information technology," he said.
Now, he doesn't anticipate using the plan in earnest until fiscal 2004. However, Missouri already is applying MoVAP's TCO assessment tools to statewide HIPAA compliance efforts. Wethington also expects to use MoVAP on upcoming projects for online business registration, auto registration renewal and professional licensing.
O'Donnell acknowledged that it's neither easy nor quick for government jurisdictions to assess current cost structures and devise meaningful ROI measures.
"You really have to ask yourself some difficult questions -- as a state, as an agency head and as part of a committee," O'Donnell said. "It takes some creativity and paradigm shifting to really get down to what an ROI might be. You've got to be willing to confront that."
Both Missouri and Wisconsin say they're trying to fashion data-collection processes that avoid unneeded complexity.
"In the past, we've had a return on investment, cost/benefit analysis methodology. The issue was that it was viewed more as just red tape. It wasn't a critical-thinking analysis," Meyer said. "So we'll be looking at this in a collaborative fashion to make sure we're not doing something that's just an additional bureaucratic step."
Despite the difficulties, states implementing TCO/ROI programs expect to reap huge benefits. Not only do these techniques give jurisdictions a means to justify continuing e-government investments, they also yield superior systems, according to Wethington.
"As you insert metrics into any process, I think it improves that process," he said. "That ultimately leads to better projects and better results."
Indeed, paying attention to ROI throughout a project's lifecycle -- from planning, to funding, to implementation, to operation -- often results in the sorts of enterprise IT systems that help jurisdictions avoid costs long after the project is complete, O'Donnell contended.
For example, this sort of analysis shows the benefit of enterprise payment engines that handle monetary transactions for multiple online services, she said. Although the enterprise technology may cost more to implement than a single-agency solution, it becomes the more cost-effective alternative as agencies throughout the state plug in.
Up-front planning required by ROI programs also results in IT infrastructures that accommodate new e-government functions more easily and inexpensively. "If you've thought it through at the front end, then you've built a scalable, flexible architecture to service the needs of the state," O'Donnell said. "All you have to worry about is the incremental functionality for the applications."
And, perhaps most importantly, ROI programs ultimately equip government officials to show concrete results from e-government spending.
"I think you need to demonstrate that you are either saving money or you are improving the quality of life for citizens," said Wethington. "It has to be something that you can actually quantify, and you've got to do it in a manner that allows you to be consistent."