Yet despite ITPM's promise as a tool for aligning IT investments with public policy, public-sector CIOs are struggling to reap its benefits.

For example, nearly 10 years after Congress passed the Clinger-Cohen Act, which requires federal agencies to improve the way they acquire and manage IT, just one in three agency executives believe they are measuring and controlling IT spending in compliance with that legislation. Anecdotal insights from state and local agencies suggest the situation there is much the same.

This discovery is the result of research on ITPM conducted by DiamondCluster, the Kellogg School of Management at Northwestern University and George Washington University, with more than 50 senior IT executives working in the federal government. The research team defined ITPM as the combination of tools and methods used to measure, control and increase the return on both individual IT investments and on an aggregate enterprise level.

A portfolio includes all direct and indirect IT projects and assets, including components such as infrastructure, outsourcing contracts and software licenses. Insights from this research are applicable to any public-sector CIO charged with increasing value and reducing risks on individual and aggregate technology investments.

ITPM has helped CIOs combat acute budget pressure and skepticism in addition to high expectations for IT performance and accountability. Because of intense scrutiny about spending and fierce competition for cash, few investment proposals these days stand a chance without a solid business case. There is also a general desire for more information; stakeholders expect accountability and want a clear understanding of how spending relates to results. And based on high-profile failures of expensive IT projects -- various customer relationship management and enterprise resource planning initiatives, for instance -- there is a lingering wariness about the possibility of IT investments' returns.

The public-sector executives we studied realize that ITPM can provide real value. They know the practice ensures that IT investments -- and there may be many -- all support an organization's overarching goals. They are also aware it provides a method for determining whether projects should be abandoned, and that it helps mediate issues about where to allocate resources within organizations.

Yet these public-sector executives still don't believe they are adopting ITPM measures effectively. And they are not alone.

Previous research and client work has convinced us that the problem in adopting ITPM is widespread, if not universal. State and local governments, as well as corporations of all sizes, struggle with the same issues.

During our study, we asked: If public-sector CIOs know they need to adopt ITPM techniques, why haven't they? What specific problems crop up? How can problems be overcome?

What Goes Wrong

The problem is not desire. The issue lies with execution -- a wide gulf exists between what executives wish to complete and what has actually been done. The largest gaps are in the ability to create a portfolio and in using information to make better decisions.

When executives were asked what stands in their way, they said they face the biggest hurdles in internal politics, estimating the real benefits of IT and insufficient knowledge of certain financial concepts.

By diving deeper into the data, we found that problems with ITPM generally occurred in three distinct areas: skills, processes and attitudes.

Skills: In many cases, IT departments lacked the skills needed to implement ITPM effectively.

About 46 percent said they lack deep financial analysis skills. "I need to build up the capabilities of my staff to better flesh out business cases and requirements," said one respondent. "That work ... needs to be more rigorously thought out and more structured."

Sixty-four percent said they "have difficulty calculating the benefits of IT." One executive explained that his or her department can measure project health -- assessing costs and major deliverables -- but that doesn't address the significant issue of what contribution a project is making to the productivity of operations.

Processes: Detailed processes were often lacking as well. "Having a strategic plan is one thing," one executive said. "Following that strategic plan and building tactical plans based on it is another."

Where did plans break down? There was often a lack of process in place to foster governance or accountability. Fifty-four percent said they "take no corrective action when benefits are not delivered," and 47 percent said they "have more than one governing body for IT investment decisions."

Even when processes were in place, they could be of limited value. "We have funds we know we're going to spend on something," said one executive, "but we don't necessarily always know what it's going to be until the year is in progress ... How do you do portfolio management in that context?"

Attitudes: The executives we studied also struggled with problematic attitudes. In about half the cases, a lack of agreement existed between leadership and staff about how to implement ITPM. While most agreed on where they wanted to end up, they could not agree on how to get there.

In addition, several executives cited "change management" as a large barrier to implementing ITPM. "People in a lot of organizations don't understand the need to capture costs at a detailed level, [which ITPM requires]," said one. "Getting them to change the way they do things is a hurdle."

What You Get When ITPM Goes Right

Regardless of the challenges these executives faced, they saw significant and tangible benefits from adopting ITPM. Improved decision-making, having a holistic view of technology assets and better communication topped the list of improvements respondents listed in our survey. Other benefits included:

  • improved visibility into the IT organization;

  • ability to objectively choose best projects for funding;

  • clarity and confidence about investment decisions;

  • assurance of strategic alignment;

  • the ability to better serve the public and provide better outcomes;

  • cost reduction for overall IT; and

  • budget justification.

    In one-on-one conversations, respondents cited specific cases of financial and operational improvements. One executive explained that the reporting associated with ITPM provides enough transparency in the decision-making process that an organization can now avoid taking a project so significantly over budget that it reaches the point of no return. Another mentioned that the CIO could now respond to the common question: How are you spending your money? He or she could also explain why the money was being spent. "The 'why' is something a lot of organizations have trouble explaining," the CIO said.

    Other respondents pointed to the education of internal executives as a key benefit to the ITPM process. "[ITPM] allows us to communicate better with the business side of the organization. Many managers don't understand what IT does and why we benefit from it," said one. Another reported, "In the last two years, our division directors finally came to recognize that the accountability and the financial responsibility for the success of the projects resided with them. So they have been engaged in the process."

    How does an organization go from fighting challenges to reaping ITPM benefits? We found that the organizations that implemented ITPM successfully adopted a set of best practices, and many followed an iterative process when moving toward increased sophistication. In other words, these organizations did not accelerate from having no ITPM program to having a productive program in one simple step. They generally followed a phased approach that took into account the organization's specific culture and capabilities.

    Best Practices

    Our study showed a connection between adopting a specific set of ITPM techniques and improved performance. Those executives who reported enhanced performance since implementing ITPM were more likely than those who did not report improvements to have:

  • integrated IT projects with systems architecture (85 percent vs. 56 percent);

  • integrated the IT portfolio with the business portfolio (75 percent vs. 36 percent);

  • tracked/monitored benefits after projects ended (75 percent vs. 13 percent); and

  • maintained a centralized project management office (65 percent vs. 23 percent).

    The research also showed that organizations generally fall into one of three sophistication levels as they progress with implementing portfolio management.

    At the least mature stage of ITPM adoption, organizations typically have developed and instituted fundamental procedures needed to document and view specific details about IT investments. For example, they have devised methods for collecting, tracking and analyzing the cost and schedule of IT projects; this data can be compared to project estimates so corrective measures can be taken if a project isn't meeting expectations. In addition, information about IT projects and assets is tracked in a central database. A defined and documented method has been created for screening, evaluating and prioritizing IT investments. These investments -- viewed as part of a portfolio supporting the organization's overall goals -- are approved by weighing their risks and returns compared to other IT investments. Senior executives reprioritize the portfolio and force funding reallocation.

    As organizations move toward the ITPM maturity curve, they become more sophisticated in using the information to evaluate investments. They use earned value analysis, for instance, as a key financial metric for managing technology projects. IT projects are prioritized in ranks by using a weighted score that takes into account factors, such as cost and alignment with the organization's goals. As a whole, IT becomes a more critical part of business strategy and program development. Organizations in the intermediate stage also look externally to evaluate opportunities for interoffice cooperation/synergy for each technology project.

    At the most advanced stage, organizations do more to manage risk and assess the value delivered by both individual projects and IT investments as a whole. IT investments are placed into a portfolio where risks and returns can be managed on an aggregate level. Automated software or other advanced techniques are used to track and monitor the portfolio. In addition to cost and schedule data, benefit information is captured and analyzed to make sure that projects are delivering as planned; and if not, adjustments are made. Risk data is collected, tracked and analyzed for each project so that risks can be mitigated. IT takes on an even more strategic role in these organizations by driving the strategy and mission of new projects. The organization also uses a ranking system to determine which investments are funded and which are not.

    In evaluating an organization's ITPM prowess, it is also valuable to measure how far along an organization is in developing the four key components of successful ITPM: using established metrics and methodologies to actively manage a portfolio; obtaining a centralized and holistic view of IT assets; using well defined criteria to prioritize investments into a portfolio that aligns with organizational goals; and aligning business and IT strategy while conforming to overarching architectural plans.

    From there, organizations can design a strategy roadmap to address its specific weaknesses, and begin to reap ITPM's benefits. In some cases, this process can transform the organization and offer substantial financial and operational advantages.

    Ultimately ITPM offers a way of determining if one is spending money in the most prudent ways. And surely that is what government must do: Use technology to do more with less.

    John Erik Garr  |  Special to Public CIO