Companies that sell security software are not bound by the same professional practices as academics, whose studies tend to refrain from sweeping estimates. Even when corporate reports involve academic researchers, the results can be suspect. Industry-sponsored studies — pharmaceuticals are an example, according to a 2003 study published by BMJ (formerly known as the British Medical Journal) — can have a bias toward the industry's economic interests. Unlike academic journals, which use a peer review process, there's no formal system of oversight for studies published by industry. The economic interest of security companies is clear: The greater the apparent threat, the greater the reason to buy their anti-intruder software. Norton, which is owned by Symantec and sells a popular suite of anti-virus software, advises in its latest cybercrime report: "Don't get angry. Get Norton."
Computer scientists Dinei Florencio and Cormac Herley, who work at Microsoft Research, the software giant's computer science lab, recently wrote a paper, Sex, Lies and Cyber-crime Surveys, (PDF) that sharply criticized these sorts of surveys. "Our assessment of the quality of cyber-crime surveys is harsh: they are so compromised and biased that no faith whatever can be placed in their findings," their report said. "We are not alone in this judgement. Most research teams who have looked at the survey data on cyber-crime have reached similarly negative conclusions."
Julie Ryan, a professor of engineering management and systems engineering at George Washington University, co-authored a paper, The Use, Misuse, and Abuse of Statistics in Information Security Research (PDF). In an interview with ProPublica, she said: "From what I've seen of the big commercial surveys, they all suffer from major weaknesses, which means the data is worthless, scientifically worthless. But it's very valuable from a marketing perspective."
Yet corporate cybersurveys are repeatedly invoked; the NSA's Alexander is merely among the most prominent senior officials to do it. ProPublica provided the NSA's media office with links to critical studies, stories and blog posts about the Symantec and McAfee numbers and asked whether Alexander or the agency was aware of them or, alternately, had other data to support the numbers he cited. The NSA media office responded: "The information is publicly available and was appropriately sourced."
* * *
McAfee was founded by John McAfee, a software engineer who wrote some of the first anti-virus software in the 1980s. The company grew quickly, thanks in part to a novel marketing strategy in those days — McAfee gave away its software, charging only for tech support. The company went public in 1992 and remained a leader in its field; last year it was acquired by Intel Corp. for $7.68 billion. "We have had just one mission: to help our customers stay safe," McAfee says on its website. "We achieve this by creating proactive security solutions for securing your digital world."
In 2008, McAfee decided to commission a report that would look at how the global economic downturn was affecting data theft against companies. McAfee put one of its public relations officials, Viveros, in charge of the project. Viveros, in a phone interview, said a technology marketing company was hired to create and distribute a survey to about 1,000 information and technology executives across the globe. Purdue University's Center for Education and Research in Information Assurance and Security, headed by Spafford, analyzed the survey results, conducted follow-up interviews and helped write the report. McAfee confirmed that it helped steer $30,000 from a foundation to Purdue for the work.
The 31-page report found that the companies surveyed had an average of $12 million worth of sensitive information stored in offshore computer systems in 2008, and that each lost an average $4.6 million worth of intellectual property in 2008. The report was released on Jan. 29, 2009, in Davos, Switzerland, during a meeting of the World Economic Forum. McAfee issued a news release to announce it, and the release included dramatic numbers that were not in the report.
"The companies surveyed estimated they lost a combined $4.6 billion worth of intellectual property last year alone, and spent approximately $600 million repairing damage from data breaches," the release said. "Based on these numbers, McAfee projects that companies worldwide lost more than $1 trillion last year." The release contained a quote from McAfee's then-president and chief executive David DeWalt, in which he repeated the $1 trillion estimate. The headline of the news release was "Businesses Lose More than $1 Trillion in Intellectual Property Due to Data Theft and Cybercrime."
The trillion-dollar estimate was picked up by the media, including Bloomberg and CNET, which expressed no skepticism. But at least one observer had immediate doubts. Amrit Williams, a security consultant, wrote on his blog a few days later, "$1 trillion a year? Seriously? Where the hell did the figure come from? To give you some perspective of size the total US GDP is about 14 trillion and that includes EVERYTHING."
The news stories got the worried attention of some of the report's contributors because McAfee was connecting their names to an estimate they had no previous knowledge of and were skeptical about. One of the contributors, Augusto Paes de Barros, a Brazilian security consultant, blogged a week after the news release that although he was glad to have been involved in the report, "I could not find any data in that report that could lead into that number. … I'd like to see how they found this number."
When the number was announced in 2009, McAfee provided no public explanation of how it was derived. "Initially we were just going to do the report, but a lot of people were asking us what was the total number, so we worked on a model," said McAfee's Viveros. This week, in response to queries from ProPublica, he disclosed details about the methodology. He said the calculations were done by a group of technology, marketing and sales officials at McAfee and were based on the survey responses.
"McAfee extrapolated the $1 trillion … based on the average data loss per company, multiplied by the number of similar companies in the countries we studied," Viveros said in an email.