Clickability tracking pixel

Security Benchmarking Could Help Government Assess Benefits of Cloud Computing Providers

Many government agencies still have concerns about SaaS and cloud computing security.

by / April 15, 2010
Previous Next

Despite growing mandates for government agencies at all levels to adopt cloud computing, many agencies understandably question the benefits and weigh the security risks of making a move to this newest brand of outsourcing. A historical perspective can help shed light on the pros and cons of making such a decision.

In 1799, a stockbroker named David Ricardo read Adam Smith's economics classic, The Wealth of Nations. The book fueled Ricardo's profound interest in economics, which was reinforced by his friends and fellow economists Thomas Malthus and James Mill. In 1817, Ricardo published his own classic, On the Principles of Political Economy and Taxation, in which he outlined one of the most significant insights in modern economics: the theory of comparative advantage.

The theory helps answer profound questions like, "Should Bill Gates mow his own lawn or use a lawn service?" The answer, of course, is that he should use a lawn service. As Ricardo explained, the answer is the same even if Gates has an absolute advantage in lawn mowing; that is, he can mow his own lawn for less money, can do it faster and can do a better job than the lawn service.

Comparative advantage looks at opportunity costs, not absolute costs. If Gates mows his lawn, he will have less time to do more profitable things, such as build his company. He probably values his nonproductive personal time more than the cost of the lawn service. Gates has a distinct advantage because the cost of the lawn service is known. He can compare the cost of the lawn service with the imputed cost of his effort to do the same job. Thus, he can make an informed decision to use the service or do it himself. As Ricardo's theory explains, the transaction between Gates and the lawn service profits both parties because they each gain economic advantage from the transaction.

Most government agencies have no comparative advantage in software capabilities, such as customer relationship management, enterprise resource planning or data warehouses. They're not core functions of the organization. Therefore, there's an opportunity cost to investing in these noncore capabilities. For many years, government agencies had to do the equivalent of "mowing their own lawns" for all their applications. There was no service for hosting and maintaining noncore software applications. Now with software-as-a-service (SaaS) alternatives, public-sector agencies and commercial companies can take advantage of the comparative advantage of the cloud - using Salesforce for constituent, case, grant or partner management; using Google Enterprise for e-mail; or other cloud software vendors and services.

There's another important point: With usage-based pricing, these organizations can now assign a value to the service. To continue the metaphor, where at one time organizations had to mow their own lawns, they can now use lawn service and know its precise cost. Therefore, government agencies can make an informed, value-based decision. Organizations may not choose to use a SaaS or cloud solution, but if their competitors do, they may reap an advantage. If all but one agency head uses a lawn service, the group will reap a comparative advantage that the one holdout will have to make up for in some way.

There's another principle that Ricardo outlined in his work: Specialization and division of labor increase efficiencies and decrease absolute costs. A SaaS provider, with the efficiencies of a multitenant environment, can usually provide an application at lower cost than a company can develop and operate in-house. Furthermore, the cost and quality advantage will likely increase over time since the specialist is likelier to innovate to maintain an advantage over competitors. As the noted economist Russ Roberts pointed out, "Trade creates wealth. Self-sufficiency is the road to poverty. ... What is wise and productive in one time and place may not be wise and productive in another."

What About Security?

There are compelling advantages to using SaaS: Capital becomes available to spend on core capabilities, costs become predictable, the service improves over time, etc. So why isn't everyone flocking to SaaS? The No.1 concern is security, cited by more than 60 percent of IT executives surveyed from Fortune 500 companies and large government agencies. A study by Kelton Research found that a 5:1 ratio of companies trust internal IT systems more than cloud-based technologies due to fear of security threats and losing control of data and systems.

Is this fear of losing control rational, or should the comparative economic advantage in cloud computing also translate into a comparative advantage in security? Despite widespread reports of qualified information security staff shortages, organizations seem to think they have a comparative advantage in security. Like Garrison Keillor's Lake Wobegon, where "all the women are strong, all the men are good looking, and all the children are above average," nearly all organizations believe they can better secure their data than an outside organization.

Cloud security concerns are not necessarily irrational. Cloud computing is the new big thing. The absolute number of companies offering cloud computing solutions is staggering; it's comparable to a newly deregulated industry, such as the telecommunications boom in the late-1990s. Like the telecommunications industry, a majority of these new cloud computing entrants are expected to fail. Struggling organizations cut corners, particularly in their security controls. Agencies must know what might happen to their data if their cloud provider goes out of business.

Just a few years ago, there was no way to purchase SaaS or to benchmark cost compared to in-house applications. And until recently, there was no good way to benchmark SaaS security against in-house applications. Now key decision-makers can make informed, risk-based choices about the security of their cloud computing providers.

There are at least a few ways to benchmark SaaS security. One is for an organization to compare the SaaS provider's security controls to its in-house controls. This can be difficult; while the granting of federal certification and accreditation is public knowledge, the actual findings and implementation details of security controls generally are proprietary. Neither the SaaS vendor nor the organization that's considering a SaaS solution is readily willing to divulge this information to the other.

Security Benchmarking: The Short and Quick Way

I once worked with a security graybeard who had done more than his share of security assessments. He confidently claimed that he could evaluate the security of an organization in about an hour. His method was to look at two security areas that are not only difficult to implement, but are the first to degrade in an organization: auditing for misuse and contingency planning. Auditing for misuse is something no organization likes to do. First, it involves sorting (often manually) through stacks of log files. Second, conducting an audit implies that some staff members are untrustworthy. True contingency planning involves testing failover and therefore risks downtime. It also requires a lot of planning and after-hours work. Eventually organizations often opt for "tabletop" tests that don't risk their operations, but also don't truly test their ability to recover in a major outage.

Organizations that perform solid audit and contingency planning will have excellent all-around security programs.

Security Benchmarking: The Moderate Evaluation

A federal agency asked my company to help it evaluate the security controls of a large commercial SaaS provider for one of its applications. Together we came up with a list of several key control areas they were most interested in evaluating from a benchmarking standpoint. The list was as follows:

  • Information commingling. Cloud computing platforms use multitenant environments where customer information is logically but not physically separated. How cloud providers logically segregate customer information is critical. Unfortunately these methods are proprietary, and vendors are often reluctant to divulge this information to their customers. Under nondisclosure agreements, a potential customer may be able to determine this information.
  • Personnel screening. Personnel are always a weak link in a system's security. Cloud platforms are a potential gold mine of information for hackers. The easiest way to get access to information is through an insider. Therefore, it's imperative that cloud providers put their employees through rigorous background checks before they get access to back-end systems.
  • Application/vulnerability assessment testing and audit processes. Application-level attacks are becoming prevalent. This is a concern for SaaS providers, so it's important to evaluate what kinds of forensic capabilities the provider can offer in case of a security breach.
  • Security event/intrusion detection monitoring. Cloud platforms are accessible to the public Internet and therefore will be subjected to a large number of attacks from around the world. The amount of high-value data they contain also increases the likelihood of sophisticated attacks. Therefore, it's important that cloud providers have best-in-class monitoring systems.
  • Business continuity processes. As mentioned earlier, true testing of contingency plans, at least once per year, is an important part of ensuring availability. Also, treat the viability of the cloud computing provider as a business continuity issue. What's the likelihood the cloud provider will be around for the long term? What are the options for retrieving data if they don't survive?
  • Compliance with IT security policies and standards. Maintaining and following policies, procedures and standards is often seen as a formality, but it's an important indicator of a disciplined organization. Ask to see the cloud provider's policies and procedures. Have they been updated lately or are they treated as shelfware?

Identifying and defining system boundaries. It's important to know the data's physical and geographic location. Different countries have varying standards for data security and privacy. This includes the ability for law enforcement agencies to seize information, which can be very difficult in a multitenant environment. Second, it's important to understand any interfaces. Some larger SaaS providers work with numerous third-party application providers. While these third-party applications can be useful, they may not adhere to the security controls of the larger SaaS provider. As such, they must be scrutinized because they potentially can be back doors into customer data. Third, having a secure cloud provider doesn't guarantee the security of customer data.

Security Benchmarking: The Full Evaluation

Implementation of security controls can be evaluated against established control standards. International Organization for Standardization (ISO) standard 27001 and National Institute of Standards and Technology Special Publication 800-53 are two control sets that are widely accepted in the private and public sectors. As an example, Salesforce is ISO 27001-certified and has an accreditation (authority to operate) with a federal agency. Other SaaS providers are quickly realizing they must follow suit to gain the trust of larger customers. In fact, Federal Information Security Management Act authorization is a prerequisite for all federal SaaS customers.


SaaS offers compelling advantages for public- and private-sector organizations. It frees up capital to spend on core capabilities, costs are predictable, and specialization and competition mean the service will improve over time.

IT managers have legitimate security concerns when deciding to use outsourced SaaS applications. However, security need not be the fly in the ointment that stops organizations from using SaaS for their noncore applications. Straightforward security benchmarking of cloud providers can give managers the comfort they need to take advantage of the benefits.

Let someone else mow your lawn. Self-sufficiency is the road to poverty.


Looking for the latest gov tech news as it happens? Subscribe to GT newsletters.

Michael Wojcik Contributing Writer
Michael Wojcik, a certified information systems security professional, is a manager in the Risk and Compliance Practice of Acumen Solutions Inc., where he works on cloud security issues. He successfully led a team that shepherded software-as-a-service provider Salesforce through its first Federal Information Security Management Act accreditation.
E.REPUBLIC Platforms & Programs