Energy Hogs on the Server Farm

CIOs must start paying closer attention to their data center energy bills -- they're big ... and getting bigger.

by / December 19, 2006
In his work helping organizations improve data center efficiency, Steve Kaplan's first questions are how much energy they use and how much it costs.

"The CFOs [chief financial officers] don't know. The CIOs don't know," said Kaplan, president of consulting firm AccessFlow Inc. in Benicia, Calif. "They just accept it as a cost of doing business and pay blindly."

But energy consumption is becoming a much more high-profile issue for CIOs as it constrains data center growth. A 50,000-square-foot data center uses approximately 4 megawatts of power, or the equivalent of 57 barrels of oil a day. Although no one really knows how many servers are spinning in the United States, research firm IDC estimated there were 1.8 million new servers installed in 2002, and by 2009, that number will increase to an estimated 4.9 million per year.

With electricity costs rising, the expense of powering and cooling that equipment is becoming more of an issue than server purchases themselves.

Early in 2006, Sunnyvale, Calif.-based chipmaker AMD surveyed 1,200 data center customers, and 83 percent of the respondents identified power as a top concern. Yet fewer than 30 percent of those companies have an action plan to tackle the issue.

Even legislators are starting to show concern. Noting that the annual cumulative energy cost for servers and data centers in the United States is approximately $3.3 billion, U.S. Rep. Anna Eshoo, D-Calif., introduced legislation in July calling for the Environmental Protection Agency to research ways to reduce data center electricity consumption.

Public-sector IT organizations are getting squeezed from two directions. Information previously stored in courthouse file cabinets must now be accessible online around the clock. The push for more Web-based services has led to the rapid growth of power-hungry data centers, while at the same time governmental entities are making commitments to reduce energy consumption and greenhouse gas emissions.

Yet until recently, IT executives have had little awareness of the problem, said Richard Hodges, principal at GreenIT, a Sonoma, Calif.-based consulting firm that helps IT organizations focus on environmental sustainability.

"Only in the last six to eight months have CIOs started to look at ideas to reduce energy consumption in the data center," Hodges said. "It's tough to get this to be part of their mindshare, because they don't pay the bills. If CIOs were cross-charged for 25 [percent] to 30 percent of the energy bill, they'd pay attention to it. It's a result of siloed organizations. IT and facilities can no longer afford not to talk to each other."

Gold Camp Seeks Efficiencies
In California, the state's ambitious plans to cut greenhouse gas emissions have put P.K. Agarwal in the spotlight. As director of the state's Department of Technology Services (DTS), Agarwal oversees an agency that spends $2 million a year on energy in its data centers, and is seeing demand for its data services to grow by 15 percent to 20 percent per year. The DTS, however, is not shying from a goal set in an executive order by Gov. Arnold Schwarzenegger to reduce state government's energy consumption by 20 percent by 2015.

"During last summer's heat wave, there were 20 days when the state was consuming power at levels not expected until 2011," Agarwal said, "so there's a great sensitivity to this issue."

Schwarzenegger created a "Green Action Team," a Cabinet-level task force to implement sustainable building practices and energy-conservation efforts throughout California.

"With such visible sponsors and a commitment at the top, there'd be no place to hide from this issue even if we wanted to," Agarwal said with a laugh. "We have a chance to be a role model on this issue, so what better way than to set some aggressive targets."

Agarwal said he believes that through purchasing more energy-efficient servers and retrofitting buildings, the state's IT infrastructure can cut energy use, despite the increased demand.

As a result, in September 2006 the state's green building team began an "energy diagnostic test" on the 133,000-square-foot Gold Camp data center, which houses about 800 servers and 16 mainframes in Rancho Cordova, Calif. After studying servers and uninterruptible power supply equipment, chiller loading, airflow, staff training and data center design, they will make recommendations for retrofitting the building for energy efficiency improvements.

"For a long time, there was a mentality that computers were expensive and electricity was cheap," Agarwal said. "Now computers are getting cheaper and electricity is expensive, so we're rethinking how we look at our infrastructure."

Fighting Server Sprawl
Another IT executive who knows exactly what he's paying to power his data center is Kevin Haggerty, information services director of Carson City, Nev.

"Our energy bill has averaged $40 per server per month," said Haggerty, adding that the annual bill for the 35-server center was between $15,000 and $17,000.

The fast-growing city of 56,000, near Lake Tahoe, is feeling the pinch of increasing energy costs and growing demand for computing power from applications such as GIS, he added. To fight back, the city is in the midst of "virtualizing" its data center's servers using software from VMware and four of IBM's new line of X366 servers.

Most organizations experience "server sprawl" because every time they get a new application, they run it on its own server. So in the average data center, the servers are running at 4 percent to 8 percent capacity. Instead of assigning each server an application, virtualization lets users divide individual servers into multiple "virtual machines," increasing their efficiency and shrinking the data center's footprint. Many of Carson City's old servers, assigned to an individual application, were being used at only 5 percent to 10 percent capacity. "I want them spinning, not sleeping," Haggerty said. "The goal is 70 [percent] to 75 percent utilization."

Through this virtualization process, Carson City expects to shrink its annual energy cost to less than $3,000 per year.

"A very big part of this project's return on investment [ROI] is energy and cooling," Haggerty said. "Our finance director was more than willing to help get those numbers together to make it pencil out."

The biggest projected savings is to avoid buying more servers. Power was No. 2 on the ROI list, he explained. "Each old server chewed up 500 watts of power. The new IBM servers might consume 700 watts, but I am replacing 15 of the old servers with one of these." The IT energy reduction in Carson City is part of a larger citywide effort to reduce energy consumption and costs. For instance, Haggerty's office building, City Hall, is expected to install solar energy panels in the next year.

Looking to the Vendors
John Jacobson, Seattle's newly appointed enterprise computing services manager, was only in his post a few weeks when he received a memo last summer from the city's Office of Sustainability and Environment asking if the IT staff could research ways to reduce greenhouse gas emissions from computer equipment in municipal operations.

"They realize we have a building full of energy-sucking equipment here, so they thought this would be a good place to look for reductions," Jacobson said.

The city's 9,000-square-foot data center has gradually grown to 450 servers, and the city's demand for online public services keeps growing. Yet the city is also an environmental leader among municipalities. Mayor Greg Nickels has committed Seattle to making deep cuts in its greenhouse gas emissions to match or even exceed the Kyoto protocol.

The memo prompted Jacobson and his data center colleagues to study their energy bill. They found that through the first eight months of 2006, the data center had consumed 2.7 million total kilowatt hours at a cost of $135,000, almost 44 percent of which was for cooling.

"We have been working with our existing vendors to try to get them to define a road map adorned with green, energy-efficient servers," he said, adding that the computer manufacturers' responses have been limited to encouraging the use of more blade servers, which allows for more servers in a smaller space, and virtualization. The vendors he spoke with also expressed hopes that new CPUs from Intel and AMD will make the server environment more energy-efficient.

Jacobson said he fears that unless the vendors work to solve the problem, his office won't have much to offer the city in the way of energy reductions.

Driving Awareness
Power is becoming a major concern for Michigan as it begins planning its five- to 10-year outlook for data centers, said Pat Hale, the state's director of technical and data center services. The state supports more than 800 applications for 19 state agencies.

CIOs have traditionally thought of buying more servers as a commodity decision. But Hale said "dense computing" using blade servers is creating an environment that requires physically modifying buildings to get more airflow. "You get to the point where keeping the servers running costs as much as buying them," he said. "Michigan will be at that point in about three years, and that will help drive that awareness up to the CIO's office -- when their decisions about servers force facilities to spend significant amounts to accommodate those systems."

Hale added that decision-makers who do not consult closely with facilities on server purchases are making a mistake that could cost them millions of dollars.

Michigan just went through a data center consolidation, moving servers from 22 smaller data centers into three of its largest ones. "We've retired 30 percent of our servers instead of moving them. The power savings was one of the intangibles," Hale said, "but we are doing more computing with fewer servers." Like Seattle's Jacobson, Hale said the pressure on the vendors is palpable to come up with more energy-efficient designs.

Vendor Focus
Some chip and server vendors are responding to data center managers' concerns about energy efficiency, and simultaneously trying to raise awareness about the problem at the CIO level.

Chipmaker AMD has helped launch an organization called the Green Grid, which the company hopes will provide a reference point for IT managers and vendors seeking to share best practices in data center power management.

One goal, said Brent Kerby, AMD's product marketing manager for Opteron chips, is to develop a clear measurement of power consumption.

"We need to provide a good metric to go along with a performance score, so buyers can see power consumed," he said. "One chip might do 5 percent lower on a performance test but might consume 20 percent less power, and buyers may want to make that part of their purchasing decision."

Five years ago, Sun Microsystems started to focus on energy efficiency in its server designs. The Santa Clara, Calif.-based company created a Web site that lets customers estimate their energy savings from switching to newer, more energy-efficient servers.

It also developed a rebate program with California utility Pacific Gas and Electric, which Sun's Vice President of Eco Responsibility Dave Douglas said is the first utility rebate program involving servers.

"We noticed people could save $1,000 a year per server in energy costs for servers that cost less than $10,000 each," he said. The rebate ranges from $700 to $1,000 per server depending on the configuration, he said.

Douglas, who regularly talks to IT executives about energy issues, sees CIOs evolving through three stages. The first group, which he estimates now encompasses about 80 percent of CIOs, is focusing on practical issues and starting to run up against constraints involving space, cooling or power.

He labels the second group "energy aware." Like Carson City's Haggerty, they know what their power bill is and what utility they're getting power from.

He calls the third and smallest group "eco-aware." Like California's Agarwal, the companies or agencies in this group have set greenhouse gas emission goals, and they understand the energy implications for the environment of their data center.

"This is the evolution of the CIO," Douglas said, "from being hit by basic constraints to being moved to look at their energy bill and then to seeing the bigger picture."
David Raths contributing writer