Analytics is putting a dent in one of the highest error rates of any government benefits program.
Cook County is Illinois’ economic powerhouse. Millions of people live and work there, generating billions of dollars in wages. So it was easy to miss the modest operations of about 80 employers registered in the county with the Department of Employment Security’s unemployment insurance services. Between 2009 and 2011, these companies filed at least 900 unemployment insurance claims with Illinois, Indiana and Minnesota worth $8.7 million. All the claims were bogus; the identities of the employees either fictitious or stolen, according to newspaper reports.
The fraud ring, which included at least 14 people, is just one of many examples of what has been a nagging problem with the nation’s unemployment insurance program, a federal-state system that provides temporary income support for unemployed workers. The system is funded by taxes collected from employers and held in trust funds administered by individual states.
Unemployment insurance (UI) is one of government’s bigger benefits programs, paying out nearly $33 billion in 2016 (at the height of the Great Recession, UI pumped nearly $155 billion into the pockets of laid-off workers). But the system also has one of the highest error rates among state benefits programs, worse than Medicaid, the Supplemental Nutrition Assistance Program (SNAP, commonly known as food stamps) or rental housing assistance. In 2015, the system made $3.5 billion in improper payments, an error rate of 10.7 percent, according to the U.S. Department of Labor.
Not all of those improper payments were the result of fraud rings, like the one that authorities shut down in Cook County. In fact, the vast majority of improper payments stem from everyday people not always understanding the obligations of the system, said Scott Malm, public-sector workforce and employment leader for Deloitte. Instead, people often continue to claim benefits after they return to work up until they receive their first paycheck. “The reality is that as soon as you start working, you have to stop claiming benefits,” said Malm. “Those kinds of overpayments have to be identified and collected.”
For years, this kind of benefits leakage, whether fraudulent or due to human error, has been a drain on the UI system and the trust funds that provide the benefits. But states have begun to tackle the problem with better technology on two fronts. First, states are modernizing their UI computer systems, some of which are decades old and still run on mainframes. “Roughly half the states have launched modernized system upgrades,” said Scott Sanders, executive director of the National Association of State Workforce Agencies. The new systems make it easier to operate online reporting services for the unemployed and to collect more data that can be analyzed rapidly.
The Struggle to Modernize UI
In the past 10 years, nearly half the states have launched efforts to modernize their unemployment insurance systems. Nobody questions the need for the upgrades. Most of the systems in the country still run on mainframes, a computer technology that is not only out-of-date, but also has become increasingly expensive to maintain as parts become harder to find and as workers with mainframe skills head for retirement.
Utah should know the power of data analytics. Back in 2006, it became one of the first states to modernize its UI system at a cost of $14 million. It created a data warehouse that has allowed the agency to amass data to monitor and evaluate possible fraudulent activity. In 2015, the state paid out $200 million in unemployment benefits, with a fraud rate of 1.3 percent, well below the national average. The vast majority of the fraud — 87 percent — is related to reporting errors: unreported earnings and unreported job separations, said Beebe.
To tackle the majority of overpayments, such as unreported job earnings, the Department of Workforce Services works with a third-party vendor to cross-match what wages an individual reports while filing on a weekly basis, rather than wait for when the employer reports to the state on a quarterly basis. “With cross-matching, we are able to detect those unreported earnings, sometimes 16 weeks sooner,” said Beebe. Utah is one of only a handful of states that uses third-party data to verify wages.
The department uses analytics for the trickier fraud cases, such as identity theft and fictitious employers. Starting two years ago and working with an in-house, custom-built system, investigators have used analytics to find suspicious patterns of activity and then assign prioritization scores to the results, which allows them to focus on the worst cases.
“Some of the trends we look for might be individuals filing from particular locations, with particular contact information,” said Beebe. “Sometimes it’s a number of filings that have the same or similar information in particular data fields.”
Overall, analytics has helped the department detect fraud a lot faster than in the past. By keeping the profiling model constantly up to date, they have been able to focus on the most suspicious accounts and shut them down quickly. “In the past few years, we haven’t had any cases go for more than a few months,” Beebe said.
While Utah has kept a firm lid on fraud problems, the story in Florida is quite different. For years, it has been ground zero for identity fraud. In 2015, Florida had the highest rate of identity theft in the country, said Cissy Proctor, executive director of Florida’s Department of Economic Opportunity. Much of that theft is tied to the siphoning of government benefits, such as UI, according to the Federal Trade Commission.
In 2013, the state began to tackle the problem by upgrading its UI computer system and developing a front-end fraud detection system known as the Fraud Initiative Rules and Rating Engine, aimed primarily at detecting identity issues. While the initial effort was modest, involving Excel spreadsheets and Google Analytics, the state has since built a sophisticated data analytics operation using open source tools, such as Apache Cassandra and Python.
The new UI system, built at a cost of $77.9 million, has sped up the process for using analytics to fight fraud in real time. “Our IT team is constantly doing data analytics to determine which factors are pulling a lot of fraud out of the system and which factors maybe aren’t working,” said Proctor.
For example, the state knows it takes 30 to 40 minutes to fill out a claim, so if someone does it in seconds, the system will flag what appears to be a computer-driven claim. By identifying patterns like that, the state has caught enough fraud to start putting a dent in the problem. “Since 2013, we have stopped 115,000 fraudulent claims from being filed,” said Proctor. “If all of those individuals received full payments, it would have been almost $500 million in funds stolen from the trust fund.”
Other Florida state agencies that handle government benefits have noticed the results. The department has begun to assist several with identifying the characteristics of fraud so they can tighten up their criteria. Last year, the state dropped down to the third-worst fraud rate in the country. “When states start closing the door, they can keep identity thieves from entering their system,” said Proctor. “We are not going to completely stop identity theft, but we are taking steps to protect the trust fund.”
Unlike Utah and Florida, which have used analytics primarily as an investigative tool, New Mexico took a different approach to how it has applied technology to reduce improper UI payments. Initial steps were similar, including a modernization effort in 2011-2012 that gave the Department of Workforce Solutions the ability to cross-match data in the UI system with other information, such as birth and death records, and even names of individuals held by the Department of Corrections, to uncover improper payments to ineligible individuals. These efforts helped New Mexico reduce UI fraud by 60 percent by 2013.
But as savings from cross-matching began to level off, the department looked for new ways to get better results. One problem stood out: the amount of work needed to conduct the cross-matching. “It was a large workload. The system was very data rich, but senior management felt that we were information poor,” said Joy Forehand, deputy cabinet secretary for the Department of Workforce Solutions.
Working with a team of lawyers, labor economists, IT specialists and communications professionals, along with its partner Deloitte, the department created the Improper Payments Prevention Initiative at a cost of $1.3 million, and began to uncover trends and patterns in claimant behavior, using data analytics to find the points in the process when an individual was more likely to make a decision that would lead to improper payments.
“We wanted to focus on truly preventing incorrect and inaccurate information from getting into the system and generating an improper payment,” said Forehand. “On the predictive side, we had data on which individuals were at a higher risk of submitting incorrect or inaccurate information.”
By creating (and testing repeatedly) certain types of pop-up messages based on behavioral economics, the department began nudging individual claimants, who were considered at high risk, to provide more accurate information. For example, rather than caution individuals about the laws and penalties related to inaccurate information, the message would say that “9 out of 10 people from the county report their earnings accurately.”
Combining predictive analytics and behavioral economics is not new, but it’s the first time the technique has been used in a state government benefits program, such as UI. Pew Charitable Trusts, which has been looking at how states use data and analytics, singled out New Mexico’s work for trying to prevent the overpayment problem rather than just react to it. It also said the state’s work has paid off in terms of accuracy and effectiveness and for breaking down silos, while establishing the right kind of partnerships, both internally and externally.
“This project demonstrates a thoughtful way of using predictive analytics to save money and make a state program more effective,” said Jennifer Thornton, Pew’s manager of data as a strategic asset.
From 2015 to 2016, the state has reduced one type of overpayment from 5 percent to 2.9 percent, while the national average dropped less than 1 percent. The department expects to see a 35 percent annual reduction in this type of fraud. The system has also led to better self-reporting about the reason why a person left a job, which has helped identify individuals who were not eligible for unemployment benefits, resulting in savings on average of about eight weeks of overpayments.
The initial success that states have had with reducing improper UI payments has led to plans to expand the use of analytics. For example, several states are using analytics to deal with problems on the employer side of UI. “There’s a lot of misunderstanding of the federal and state laws that govern how a person can be classified as either a contractor or an employee,” said Forehand. Her department is developing a messaging strategy using behavioral analytics to educate employers.
New Mexico is also using the same techniques to steer the unemployed back to work faster (and reduce the time they are receiving benefits). By using analytics, states such as New Mexico hope to identify and reach out to those who could be doing more to seek a new job. “We can connect those individuals who are not returning to work quickly and have the employment side of the department help them boost their job-seeking activities,” Forehand said.
But if states expect analytics to put a bigger dent in the UI error rate, they will have to speed up modernization efforts. While half the states have launched some kind of computer upgrade, not all those efforts are complete or going well (see The Struggle to Modernize UI on page 39). The rest of the states are in some stage of planning, pre-development testing or doing something else, said Sanders.
Having a modern IT system isn’t a requirement for conducting analytics, according to Malm. “We have a clear vision of how we can deploy [analytics] even in states where the back-end systems are legacy.” But Forehand said that having a modernized UI system eased the challenge of adding in analytics. “It would have been a lot more difficult to add the analytics engine to the system if we hadn’t modernized the system first,” she said.
The universal problem, however, is the lack of funding. “It continues to be a struggle,” said Sanders. “Modernizing UI is not a top priority based on what states want to get done when it comes to technology.”
Modern technology isn’t the only requirement. Data quality is essential to a successful analytics operation. Another problem that can crop up is the number of false positives that analytics can trigger. This can occur when an agency doesn’t have enough data to use advanced analytical tools to lower false positive rates to acceptable levels, according to a 2016 report on benefits payment integrity by the MITRE Corp. Skilled analysts also are still critical to the process, and these skill sets are expensive and often hard to come by for government, according to the report.
When it all comes together, as it is in New Mexico, the improvements from a well designed analytics operation can be a game-changer. “If all 50 states did what New Mexico is doing, the savings would be in the hundreds of millions of dollars annually,” Malm said.