The company's method did not meet the standards of the Purdue researchers whom it had engaged to analyze the survey responses and help write the report. In phone interviews and emails to ProPublica, associate professor Jackie Rees Ulmer said she was disconcerted when, a few days before the report's unveiling, she received a draft of the news release that contained the $1 trillion figure. "I expressed my concern with the number as we did not generate it," Rees Ulmer said in an email. She added that although she couldn't recall the particulars of the phone conversation in which she made her concerns known, "It is almost certainly the case that I would have told them the number was unsupportable."
Viveros said McAfee was never told by Purdue that the number could not be supported by the survey data. The company moved ahead with the news release and, Viveros noted, the trillion-dollar estimate "got a life of its own."
In February 2009, President Obama ordered a 60-day cybersecurity review to look into ways to better protect the country from cyberattacks, and he appointed Melissa Hathaway, who served as a cybersecurity adviser in the Bush administration, to oversee the effort. On May 29, Obama unveiled the review and delivered his first major cybersecurity speech. The second page of the 38-page review cited McAfee's trillion-dollar figure, and the president used it in his speech, saying, "It's been estimated that last year alone cybercriminals stole intellectual property from businesses worldwide worth up to $1 trillion."
The administration's Cyberspace Policy Review (PDF) includes footnotes, and the one for the $1 trillion estimate directs readers to McAfee's news release. It is not an ordinary occurrence that a president relies on the contents of a corporate news release to warn Americans of a major threat to the homeland's economic and national security, but Hathaway, now a security consultant, told ProPublica that at the time of the president's speech she was comfortable with McAfee's estimate because it appeared to be associated with Purdue researchers. However, she became wary of it once she began making more inquiries after the speech. "I tend not to use that number anymore," she said. "I was surprised that there wasn't proved methodology behind the number."
In March 2011, McAfee published its "Underground Economies" report, which repeated the $1 trillion estimate. Criticism of it continued, too. Robert Richardson, then director of the Computer Security Institute, skeptically wrote on the group's website in the spring of 2011 that "The trillion dollar number is just too good to kill." Later in 2011, Wired's British edition reported that "if true, the figure amounts to a massive 1.6 percent of global GDP." This year, Microsoft Research's Florencio and Herley wrote an opinion piece in The New York Times that described widely circulated cybercrime estimates as "generated using absurdly bad statistical methods, making them wholly unreliable."
These critiques have now taken on added importance because government officials are citing a variety of industry-generated numbers in their efforts to bolster support for major cybersecurity legislation. The House passed its version of a cybersecurity bill this spring; the pending Senate bill, known as the Cybersecurity Act of 2012, would enable the U.S. government and private companies to more easily share information about cyberthreats and create a set of voluntary cybersecurity standards for operators of critical infrastructure.
* * *
In his speech at the American Enterprise Institute, Gen. Alexander said Symantec placed the cost of intellectual property theft to the U.S. at $250 billion a year. Tracing the origins of this statistic — as both the U.S. Government Accountability Office (PDF) and technology writer Julian Sanchez have attempted before — is not unlike pulling a piece of yarn to unravel an old sweater. Although Symantec mentioned the $250 billion estimate in a 2011 report, Behavioral Risk Indicators of IP Theft, the estimate is not Symantec's.
The report mentions the figure in passing, sourcing it in a footnote to a legal paper, where, as it turns out, the $250 billion number is not mentioned at all. Eric Shaw, one of two forensic psychologists Symantec retained to research the "Behavioral Risk" report, told ProPublica the footnote was a mistake. Instead, it should have referred to a different paper that points to a 2003 speech by FBI Director Robert S. Mueller. The figure is also cited in old FBI news releases available via the Internet Archive.
An agency spokeswoman said that although she believed FBI officials used a reliable source for the number, the FBI had neither developed the number nor claimed to have done so. She pointed to another document (PDF), from the U.S. Department of Justice, attributing the $250 billion figure to the Office of the U.S. Trade Representative.
Then-Commerce Secretary Gary Locke used the $250 billion number in a 2010 speech. Like Locke, the trade representative is a member of the president's cabinet; a spokeswoman for the office said the figure was not from them. "Your inquiry appears to refer to an industry-reported figure," the spokeswoman told ProPublica, pointing to a U.S. Chamber of Commerce paper on intellectual property theft. Sure enough, there's the $250 billion again — this time attributed to none other than the FBI.
There are other concerns about Symantec estimates cited by Alexander. Drawing from the 2011 Norton Cybercrime Report, Alexander put the direct cost of cybercrime at $114 billion and cybercrime's total cost, factoring in time lost, at $388 billion. The report was not actually researched by Norton employees; it was outsourced to a market research firm, StrategyOne, which is owned by the public relations giant Edelman.
StrategyOne surveyed almost 20,000 people in 24 countries, asking them to report whether they had experienced cybercrime and how much it had cost them. The company said it used "standard research practice for online surveys" to obtain a representative sample of Internet users. To calculate a total cost, it multiplied the estimated number of victims by the average cost of cybercrime in each country.
But that still leaves room for uncertainty, several researchers told ProPublica. For example, if responses came mainly from those most concerned about cybercrime or from those who suffered the biggest losses, it could inflate the average cost. And one person's estimate of the financial damage from a cybercrime might be completely different from the next person's guess, even if both suffered the same crime and the same amount of lost time.
A StrategyOne spokesman, asked if the Symantec estimates could be called scientific, responded, "Yes, as much as any survey or poll that relies on consumers to estimate their losses based on recall."
Some experts say that's not good enough. "Nobody can really assess the true impact of cybercrime," said Franz-Stefan Gady, an analyst at a security-focused think tank called the EastWest Institute. "It's really the self-reporting — because we can't verify it. It's just as simple as that."
In their 2011 paper, Florencio and Herley of Microsoft Research did not specifically mention the Symantec or McAfee numbers. But they observed, "Far from being broadly-based estimates of losses across the population, the cyber-crime estimates that we have appear to be largely the answers of a handful of people extrapolated to the whole population."
Sen. Collins added another layer of confusion about the mysterious $250 billion figure when she spoke last week in support of the cybersecurity bill. In remarks on the Senate floor, she mentioned Gen. Alexander and said, "He believes American companies have lost about $250 billion a year through intellectual property theft."
Collins' office declined several requests for comment. A spokeswoman for Lieberman, who similarly cited Alexander and the $250 billion figure, replied, "Senator Lieberman and his staff believe that McAfee, Symantec, and General Alexander are reputable sources of information about cybersecurity."
This article was originally published by ProPublica.