Altering the Outcomes

These days, discussions of welfare reform invariably turn to outcomes. Historically, welfare programs used measurements that assessed process efficiencies. In an entitlement system, the measurement of error rates and the number of cases per caseworker were appropriate, since the business was focused -- as its primary driver -- on determining eligibility.


In the new world of welfare reform, with time-limited benefits and workfare programs, the objectives are more complex than getting the applicant a check. The new paradigm involves helping recipients develop enough self-sufficiency so that they do not need state-provided financial supports. It is no longer acceptable that welfare programs are just efficient; they must also be effective. The only way to confirm that they are effective is to measure their effectiveness against some agreed-upon criteria. Therefore, the discussion of outcome measures is critical to welfare reform.

William Kilmartin, comptroller for Massachusetts, who is a strong advocate of management accountability in the public sector, said, "Clarifying the outcomes we desire is important. Let me give an example of a situation that makes outcome measurement so tricky in the field of human service. Take an example of a hypothetical mother and child that were getting assistance under the old Aid to Families with Dependent Children (AFDC) program, the child was removed from the home and put into the care and protection of the state in its foster care program. The child welfare worker that removed the child from the mother's care was supposed to inform the welfare caseworker that the child was out of the household, so that the benefit to the mother was reduced. The assumed measure of program success for welfare was to reduce the caseload level; we went one case down in AFDC and one up in child welfare. But a positive outcome measure for the child welfare program was to get the children in foster care back to their mothers. Working toward that outcome, the child welfare caseworker would not have informed the AFDC program of the foster care placement, because doing so would have reduced the mother's income level. The child welfare caseworker worried that the reduced resources available to the mother would reduce the likelihood that she could create the conditions necessary for the return of the child. The finance people went nuts when this type of situation occurred, but from the perspective of the child welfare worker it was a rational decision because of the assumed outcome of reunification. From the welfare agency's perspective it was a poor outcome -- fraud!"

Kilmartin went on to say, "I think that you would find that every public servant is in vociferous agreement with the concept of performance measurement. Ten percent of workers just show up for their job; 90 percent want to perform and would not mind measurement accountability. The difficulty is converting policy objectives into metrics for accountability. A lot of people have tried to use the budget process as a tool for holding agencies accountable for outcomes, but I don't know if that works. In the private sector, goals and objectives can be converted to financial terms easily. In the public sector, it is a stretch beyond reason to think that all things can be measured in terms of budgetary outcomes."

"It sounds good in concept," stated Louis Gutierrez, chief information officer for Massachusetts, "but without energetic devotion to what they [agency leaders] are trying to achieve, measuring outcomes turns into just another administrative exercise to justify budget levels. It is easy to understand how we track activities within programs, but when it comes to tracking complex human events, we go beyond our current capabilities. Information technology (IT) can be effective in measuring program activities, but it is difficult to see how automated systems can be useful when measuring outcomes in terms of complex human events."

In early October, President Clinton seemed to suggest that caseload levels were the best indicators of success with welfare reform. He said that his administration has witnessed "the largest decline ever in the nation's welfare rolls" -- a drop of 3.6 million since he took office. The president continued by saying that this decline is "yet another piece of evidence that welfare reform is working far better than anyone had predicted it would ... this is great news for America today, and I must say that I am very, very pleased." The question of how much impact welfare reform legislation and new programs have had -- versus the tremendous strength of the economy, bullish stock and capital markets, and record low levels of unemployment -- are not answered if the only metric is caseload levels.

More complex outcome measurement will be necessary to assess how well government programs achieve policy objectives.

Gutierrez suggested that there are two possible approaches for IT support of outcome measurement. "Agencies will attempt to design measurement systems; they will be large and complex by the time they are designed to reflect all of the agency priorities. The other approach will see agencies track all of the program events from administrative systems and then store them in large [data] warehouses so that they can do ad hoc analysis of outcomes. This second approach will be more affordable but will raise some interesting privacy issues."

"The warehouse approach will require that there be common identifiers established so that we can track lifetime interactions with government -- economic transactions, human services received, education experiences, criminal justice involvement. It is not as though there is a linear progression, but in real life there are complex interactions, and to use automated systems to track data will require sophisticated capabilities," reflected Gutierrez.

Do Americans want all of their public interactions to be assimilated so that we can achieve the public policy goal of measuring outcomes in human services? Gutierrez suggested that the public and its elected leadership may be moving toward an "information socialism" that suggests "the fallacy that if we know everything we could get better control and better results."

The public sector has employed IT on a massive scale to increase worker productivity. Will we be able to use it to improve program effectiveness and prove that we have? The jury is still out on outcomes.

Larry Singer -- an expert on strategic computing with 12 years experience in the information technology industry serving all levels of government -- is president of Public Interest Breakthroughs Inc., of Vienna, Va.

December Table of Contents