Virginia Strengthens IT Security From Data Center to Desktop

Virginia unifies IT security policies and infrastructure across five major areas.

by / January 29, 2010

The fight for comprehensive IT security seems never-ending . The enemy storms government territories every day with as many types of malware as there are pebbles on a beach -- too many to deal with easily.

Yesterday, the "man-in-the-middle" attack on the county network stole credentials to the treasury funds. This afternoon, the Trojan deleted files from the governor's desktop. And tomorrow, some exotic new code no one's heard of will pop up somewhere unexpected. In these battles, CIOs and their staffs must operate like a machine: Teamwork should exist between the standards and policy writers, data center operators, network administrators, programmers, the help desk staff and others -- seamless coordination.

That's probably what Virginia's technology executives were aiming for when they implemented the Interlocking Spheres of Collaborative Protection project, a complicated endeavor with one simple goal: to unify the policies, infrastructure and culture across state IT that are responsible for keeping data secure. These spheres represent the technology groups that need a common strategy in the unending security war.

John Green, the state's chief information security officer, noted that the end result is rather impressive. "It highlights the amazing things that can be achieved when you get a group of dedicated people together," he said. "I'm talking about all of the employees and information security officers -- when you get a group of dedicated people together focused on the same goal and mission."

Banding Together

The Virginia Information Technologies Agency (VITA) led the charge with cooperation from state leadership, the agencies it serves and Northrop Grumman, which manages Virginia's government IT infrastructure and services. They focused on five areas and separated them into spheres.

The top-down sphere represents the state's movers and shakers, including the governor, state CIO George Coulter, Green, the Legislature and agency leaders -- many of whom were instrumental in creating compliance standards and urging groups to meet them. The Information Technology Investment Board, which oversees Virginia's IT reform, can withhold project funds from agencies that don't keep pace.

In the peer-to-peer sphere, employees join groups to train, share information and network in the name of better security.

In the IT security program sphere, agencies are required to develop risk assessment and management programs for systems with sensitive information.

The infrastructure sphere covers most hardware and software changes, including the opening of a new data center, and the external sphere involves educating citizens about their personal IT security.

The five spheres represent a staggering amount of work for Virginia. And there's always more to do. "Eventually who knows? It may change," said Michael Watson, VITA's director of security incident management. "We may have to add additional input as time goes on. Technology's never a static thing. It evolved into this as we went along with the process."

And the struggle for strong security is everlasting.

"Like anything else, security is a journey. We don't expect to accomplish it all at once," Watson said. "We've grown and introduced the security culture into the different agencies and the rest of the government, and helped propagate the idea that it's a priority within the state. And as time has gone on, we've developed it."

Fixes From the Ground Up

Before the interlocking spheres project began in 2006, Virginia's IT landscape looked much different. The state had more than 90 disparate IT departments within individual agencies, and 60 percent of the state's equipment was between 8 and 10 years old. Even Virginia's primary data center was a security risk. The state's auditor of public accounts reported that 17 percent of 104 agencies lacked an information security program and 63 percent had an inadequately documented program.

Virginia's leadership decided change was due. "The Legislature, the governor and various governing

officials took a look across the commonwealth," Green said. "We noticed that the infrastructure was certainly aging. There were some issues with older systems as a result of technology being out of date." Green said he suspects some of the security controls in place to safeguard those legacy systems also were antiquated.

Fortunately lawmakers and the executive branch understood the importance of security and passed legislation that became the foundation for the information security program. Among those were policies that empowered the state CIO to develop standards and procedures for security across Virginia. In 2007, the Virginia General Assembly passed legislation requiring the CIO to direct the development of these policies and procedures and gave the Information Technology Investment Board the power to withhold funds if necessary.

Thanks to these efforts, agencies had a mandate to perform risk assessments and use the policies and tools to make that happen -- activities that fall into the IT security program sphere.

Those who spoke to Government Technology about the security project said it wasn't terribly problematic getting the work done, although John Willinger, information security officer for the state Department of Behavioral Health and Developmental Services, hinted at reluctance some agencies had with the changes.

"It's just a matter of getting those agencies that have been used to doing business in a certain way to step up and say, 'OK, we do need a change,'" he said. "I think that's probably the biggest issue -- just agencies being reluctant to take the step."

The state government invested $270 million upfront to transform the robust infrastructure sphere. These changes were spurred by coordination among Virginia's leadership, VITA and other state agencies, with substantial help from Northrop Grumman, their private-sector partner.

"Northrop Grumman was integral in the infrastructure sphere," said Matt Slaight, manager of computer systems security for the company. Northrop Grumman helped with the technology consolidation, the new data center and backup facilities for support.

The primary data center, the 192,000-square-foot Commonwealth Enterprise Solutions Center, opened in July 2007 in Chester, Va., at Tier 3 capability with power from two separate substations and an alternate water supply. According to the TIA-942 standards established by the Telecommunications Industry Association, a Tier 3 data center can have maximum annual downtime of 1.6 hours and must have multiple power and cooling distribution paths. The facility houses more than 600 Northrop Grumman and VITA employees.

Soon after the data center opened, an additional remote recovery site was completed in Lebanon, Va. This facility is 101,000 square-feet, with a help desk and backup capabilities staffed in part by employees from nearby counties and cities.

"By the December [2007] time frame, we were completed with southwest Lebanon's facility, and in the March-April [2008] time frame, we executed the first full disaster recovery test for the commonwealth, utilizing both of those facilities. So it was a very fast turnaround," said Mike Elkins, Northrop Grumman's director of infrastructure services.

Virginia now has standard security tools and policies for more than 23,000 PCs, and a single, statewide network and secure Internet gateway. The state can centrally scan e-mails and endpoints for spam and viruses, a much easier task than it was in a fragmented IT environment.

"There are certainly some benefits that we've seen in our ability to measure and monitor things happening on the network -- our ability to collectively respond more quickly when something does go wrong -- and that can either be a typical security-type incident or an incident with respect to availability of some type. It gives us more visibility," Green said.

But Green's analysis could be understated. Willinger said the process was transformative. "When I say transformation, I mean from the desktop to servers to e-mail to everything. It's a huge undertaking just for our agency alone," Willinger said. "When you

look at the entire commonwealth, it can be overwhelming."

Team Efforts

Technology alone doesn't enhance security. People also do, which is where the peer-to-peer sphere comes in. This area is all about sharing the wealth -- knowledge, training and experiences -- through three groups: the Commonwealth Information Security Council, the Information Security Officers Advisory Group and the Information Security Orientation program. They're go-to zones for personnel looking to gain some security IQ.

The Information Security Officers Advisory Group is a monthly meet-up for top security officials. The group was originally meant to influence government IT projects and policy, but membership grew too large, with 100 to 150 people chiming in. Today state information security officers use the group to share information about emerging security trends, technologies and occurrences that could affect local IT.

The Information Security Council is much smaller, with roughly 12 representatives from universities and every branch of state government, including local governments, who meet monthly. This group zeroes in on policy and strategic directions that affect security.

Participants in the Information Security Orientation program learn security strategies that will make their agencies compliant with state standards.

The sphere also includes orientation and training for new information security officers. This training is open to other personnel who want to learn about compliance.

Spreading the Word

The changes brought forth by the interlocking spheres project aren't only evident in the back-end data center and networks. Changes also are apparent on the VITA Web site.

The information security incident reporting form allows state personnel to inform VITA about anything from site defacements and viruses to inappropriate use of technology and hacks. This reporting structure complies with a crucial requirement: Executive branch agencies must report these issues to the CIO within 24 hours of when they were discovered or should have been discovered.

"That form is one of the methods we use for our agencies to communicate to us that an incident has occurred," Green said, "and the mission of that form is to activate our incident response team."

VITA also helps government workers and citizens practice good computer hygiene by offering tips and resources via the online Information Security Awareness Toolkit, a site with information and software code for visitors to take if they wish. There's a video, posters, brochures, a calendar, crossword puzzles and more -- all designed to plant the seeds of secure computing in the minds of those willing to be educated. The site's advice and links connect employees and the public to outside security resources, which is defined as part of the external sphere of interlocking security.

The toolkit was developed before the interlocking spheres project began, so these educational offerings will be fine-tuned, according to Nakita Albritton, who serves in as VITA's manager of information security and continuity of operations coordinator.

"There's going to be a section [geared] toward executive information security awareness," she said. "It'll include security articles that basically give a little snapshot of some of their responsibilities and standards. It'll have presentations in there, and as we find other ways of distributing information to them, we'll add those things."

The toolkit's YouTube video, The Duhs of Security, is a quirky 13-minute production that covers basic security steps that everyone should know. Actor Garet Chester's humorous narration features a bevy of celebrity impressions -- no doubt intended to save viewers from boredom -- as they're educated about the benefits of deleting mysterious e-mails and changing passwords. The video may need some updating: Some of Chester's characterizations, like one based on Peter Falk's portrayal of Lt. Columbo from decades past, may not register with modern audiences.

Yet the toolkit will be mercurial regardless. "We're looking to develop it a lot more, and it will be dynamic," Albritton said.

And because developing robust security takes time and stamina, officials involved in the interlocking spheres project say it will need to evolve dynamically, too.

"Security is really a continuous life cycle," Slaight said. "It's never complete. The threat is always evolving. There are always new tools coming out -- new exploits, new vulnerabilities -- and it takes constantly staying up to date and on top of the threat, and adjusting the defensive posture of the infrastructure."

Willinger feels much better about Virginia's security now that the spheres are in place. "We're a lot better off now than we were three years ago, most definitely," he said. "When you feel a little more confident in your poker hand, you feel better off about what you have."


Hilton Collins

Hilton Collins is a former staff writer for Government Technology and Emergency Management magazines.

Platforms & Programs