Imagine an employee who spends 90 percent of the workday wandering the halls and taking catnaps. In most organizations, that person would be sent packing before closing time on Friday. But not so in data centers. Governments all over the world keep them packed with servers that are kind of like this hypothetical slacker - they don't live up to their potential.
"Average CPU utilization is probably somewhere in the range of 5 percent to 7 percent, maybe 10 percent to 12 percent on a server that's really being taxed," said Tennessee CIO Mark Bengel.
Just as each person in an office - productive or not - commands a desk and chair, a paycheck and benefits, each server box in a data center takes up floor space, arrives with an initial price tag, and generates ongoing costs for software licensing, power, cooling and maintenance. These costs accrue whether the box performs at 5 percent of capacity, or 75 percent.
The desire to get a much bigger return on hardware investments is one reason some government organizations turn to virtualization technology. Virtualization uses software to simulate multiple entities inside one physical server box. Each entity has its own personality and performs as though it is a real component.
Virtualization can mean, for example, creating 20 independent servers in one physical box, each with its own operating system and configuration, and the ability to run a separate application. It might mean creating a dozen desktop PCs from one server and 12 thin clients, or dividing the resources of one network infrastructure into what looks like three entirely separate networks.
Much interest in virtualization focuses on servers. One important reason data center managers create virtual servers is to consolidate facilities, running more applications on fewer physical boxes.
That strategy offers several benefits, said Gary Chen, senior analyst with the Yankee Group. "No. 1, you'll save money on hardware: You won't have to buy as many servers. No. 2, you'll save money on space." Most government agencies and private companies pay for data centers based on square footage, he said.
"No. 3, you'll also save on power," Chen added. That's good for the environment as well as for the financial ledger, he said.
Virtual servers also make it easy to troubleshoot hardware and plan for disaster recovery. Because virtual servers are simply software, if the box they're running on experiences hiccups or crashes altogether, it's easy to slide them onto another piece of hardware, leaving end-users none the wiser, Bengel said.
Spin it up
The server farm in Tennessee's state government data center hosts 350 virtual machines. If a physical server breaks down entirely, the storage area network (SAN) that houses the virtual servers can send them to a different box, Bengel said. "It gives you tremendous flexibility to perform maintenance in the middle of the day, which you normally couldn't ever do." And if a disaster hits the data center, the SAN can send any virtual server to a backup location. "I can just spin it up at another farm at another site, and everything is intact, including the IP address."
Virtual servers work well for testing updates, code changes and patches, said Rob Campbell, senior technology specialist with the Microsoft Federal Team. They also reduce the amount of hardware a person needs to lug around when demonstrating applications, he said. "If I want to set up a demo environment that has multiple servers and clients interacting, I can do all of that on top of one laptop, because each of the demo machines can run in a virtual environment," Campbell said.
A virtual server also is an ideal platform for software developers who need a machine for a few months, said Max Arnold, Tennessee's executive director of data center operations. "Once they have [the software] developed,
they no longer require that server," Arnold said. The data center can simply absorb the virtual server back into the farm. "So you're not stuck with a large investment that you no longer need."
That's just one example of the flexibility that makes virtual servers easier to manage than physical machines. Instead of requiring a computer and an operating system to run an application, you just need a few files, Chen said. "I can move them to any server that has the virtualization software installed on it, whether it's local or half-way across the world. And you can do it just like copying files."
Leading companies that offer virtualization software include VMware, Microsoft, Oracle, IBM and Citrix.
One thing virtual servers don't do well is run large, demanding applications that need to move large streams of data through the CPU quickly. "The translation between the virtual hardware and the physical will have a performance hit," Campbell said. "You should virtualize based on intended workload and the performance profile."
"If you have a large application that has a tremendous amount of I/O [input/output], whether it be a database, or an application server, virtualization may not be your best candidate," Bengel said.
More Machines, More Work
Officials in Tennessee's IT office became interested in virtualization when the state began working toward a centralized computing model. Back when client/server systems were in vogue, the volume of data passing over limited bandwidth prompted agencies to house servers in their own facilities. As network bandwidth grew more abundant and the client/server model gave way to Web-based applications, the state moved servers back into a central data center. That posed a problem for the data center: More machines meant more work for the technical team.
"I wanted to be able to do consolidation while minimizing the amount of additional staff that I had to add to my data center," Bengel said. By reducing the number of physical machines required in the center, virtualization allowed Tennessee to manage more servers with fewer employees.
In the past, a systems analyst or systems programmer took charge of 25 to 30 machines, Arnold said. "Now I have upward of 70 machines per person."
The data center runs mostly Sun Fire X4600 machines, with an average of 25 virtual servers running on each dual-core box. There's also a server farm with some single-core HP ProLiant DL580 servers, which average 10 virtual machines per box. The state uses VMware virtualization software.
One of the initial challenges Tennessee IT officials faced as they implemented virtual servers was determining which applications would run well, and which made too many demands on the software. "We had to do some experimentation to find out," Bengel said.
They discovered that a virtual machine makes a great Web server or application server and can handle a small database, he said. "It doesn't make quite as much sense with really big applications - although even with, for example, our Web servers on our [enterprise resource planning] application, which is one of the largest in the state, we've found that virtualization is working great."
Tennessee's virtual servers will soon find a new home in two data centers that the state is building to replace its 20-year-old, 70,000-square-foot facility. At 35,000 square feet apiece, each of the two new centers will house half the state's active servers, and each will serve as the disaster recovery site for the other. If one center becomes unusable, staff can quickly move applications from virtual servers in one building to the other.
"And actually, with all of our critical systems, we won't even have to move the data over," Bengel said. Each time a server in either center conducts a transaction, it will write the data simultaneously
to SANs in both facilities. "The data will be in both places, so all we have to do is recover the physical boxes and systems, not the data," he said.
3,000 and Counting
Virtual servers also are part of the data management strategy at the Collier County Public Schools in Naples, Fla. But the school district has taken virtualization a step further. Following a successful pilot program of virtual desktops in 2006, it started implementing that technology in earnest. So far, Collier County Public Schools has served up 3,000 virtual desktops to end-users, and the number continues to grow.
"Our goal with this first phase is to get up to 6,000," said Thomas Petry, the school district's director of technology. With some 44,000 students and 5,000 staff, the Collier County Schools operate a total of 25,000 desktops. District officials eventually would like 90 percent of them to be virtual machines, he said.
Collier County hosts the virtual desktops on HP ProLiant BL35p blade servers running HP VMware ESX virtualization software. The servers "push" virtual desktops, including operating systems, configurations and applications, to users working on HP dc7600 thin clients.
The main reason the school district started deploying virtual servers was to reduce the time technicians spent troubleshooting desktop machines in the field, Petry said. "It gives us more time to spend on other things besides just fixing the same old desktop problems over and over."
Because the virtual desktops reside in the district's data center, technicians at a central help desk can solve most problems remotely. Technicians also can discard troublesome virtual machines and request new ones by clicking that option on the Web site where they log in.
"To us, they're not real machines, so they're not commodities," Petry said. Technicians delete the discarded machines, or if they think the trouble may recur, they can mark certain virtual desktops for further study.
"We want to make sure we don't have trends, where the same problem is happening to hundreds or thousands of machines," Petry said. "As time permits, technicians review the machines that have been 'trashed' and see what was going on with them, why they were trashed."
The virtual desktops also offer substantial hardware savings, since thin clients don't become obsolete nearly as fast as full-fledged PCs do. "They only have to be refreshed when major thin client technology refreshes. So you really only have to refresh the technology in the data center," Petry said. If a user requires a faster computer, technicians simply assign more RAM or other resources to his or her virtual desktop - an easy software operation.
Virtual desktops don't work well for graphics-intensive applications, the kinds used by video editors or engineers, Petry said. "Those just do not play well in the virtual desktop space." But only about 0.5 percent of the desktops in the Collier County Schools require that kind of power, so this limitation isn't an issue, he said.
One strategy the district pursued for phasing in virtual desktops is grouping together users who require the same applications and have similar workloads. Each of these groups forms a user pool, and the data center assigns each group a pool of generic desktops. When a user signs on, he or she doesn't necessarily get the same desktop every time. But since all the desktops in the pool look and behave the same, the user doesn't know the difference.
The user can't customize the desktop or install extra applications, but at the same time, there's less chance a virtual PC user will make inadvertent changes that mess up desktops and drive IT technicians crazy.
"If you can get users to the point where they can use as generic a desktop as possible, and you can do this
application push - where they have the applications they need any time, anywhere - then it really reduces the workload on your technicians," Petry said.
For users whose jobs require custom configurations, the virtual desktop software provides "sticky sessions," allowing them to get the same desktops every time they log in, Petry said.
The major challenge Collier County faced in implementing virtual desktops was choosing a virtualization software package that met all its needs. "Even with the [VMware] Virtual Desktop Manager, we're dealing with small glitches," Petry said. The district is working with the vendor to iron those out.
But virtualization technology definitely works; most users don't know they're not using standard PCs, Petry said. "It's very seamless. People just seem to get in there and experience it for themselves and figure out how easy it is to use."
Merrill Douglas is a writer based in upstate New York. She specializes in applications of information technology. E-mail: firstname.lastname@example.org