Inside San Diego Supercomputing Center's green data center
- 06 November, 2007 13:00
The 80,000-square-foot building will double the size of the SDSC's facilities; besides an additional 5,000 square feet of data-center space, the expansion will house classrooms, offices, meeting rooms and a 250-seat auditorium.
Under development since 2003, the building has an energy-efficient displacement ventilation system that uses the natural buoyancy of warm air to provide improved ventilation and comfort; exterior shade devices, such as awnings, to control temperatures by blocking the sun; and natural ventilation (the windows in the building will open) to save on energy.
The SDSC also is carefully selecting the IT equipment that will populate the data center to help lower overall energy consumption and save on operational costs.
"To marry the energy efficiency of [the building] with the IT systems and understand what impact they are putting upon each machine room" will be crucial to the data-center expansion's success, says Gerry White, director of engineering services with the design and construction office at the University of California at San Diego (UCSD), which is home to the supercomputer center.
The decision to build an energy-efficient data center came down to a matter of need, says Dallas Thornton, the SDSC's IT director. Funded by the National Science Foundation and associated with the University of California network, the SDSC provides facilities for academic research on such data-intensive topics as earthquake simulations and astrophysics. It offers users more than 36 teraflops of computing resources, as well as 2 petabytes of disk and 25 petabytes of archival tape capacity on-site.
While the high-performance computers and related equipment provided by the SDSC for this research are augmented by computers that individual projects supply, all require a significant amount of power and cooling, Thornton says. The present data center has a constant load of about 2 megawatts of power, which equals roughly enough energy for 2,000 residential homes. "Being on the cutting edge of technology, we've seen a lot of [energy] load before a lot of other folks, so we've had to do something [about energy consumption] just to stay in business," he says.
An easy sell
Thanks to the center's intense power requirements and the mandates specified in Title 24 of the California Code of Regulations, which sets the standard for energy-efficient new-building construction, Thornton and his colleagues in engineering and facilities didn't face much opposition when they tried to persuade upper management that the data-center expansion should be energy efficient. And the fact that the new data center will save significantly on operating costs made the convincing even easier.
"Saving money is huge -- it all comes back to the cost of power, so for us [saving money] with a green data center really sits well" with upper management, Thornton says, although the logic behind energy efficiency should be clear to any executive team. "The No. 1 cost in running a data center is power, so if you can create ways to reduce that footprint, it should be an easy sell."
Because much of the data center's cost savings will come from the design itself, Thornton says he can't predict what kind of operational cost savings the center will gain by buying IT equipment that consumes less power than traditional computers do. Some savings are already clear, however; the center estimates it will save 40% on operating the new building vs. a traditional building because of its plan to cogenerate power locally by using waste steam to power steam chillers that will help cool the data center, he says.
The SDSC has submitted a grant proposal for IT equipment that will go into the new data center, so Thornton doesn't know yet what that will entail. However, he predicts the center will house much of the same type of IT equipment as the SDSC's existing 14,000-square-foot data center does, which includes IBM, Sun, Dell and HP servers; Sun, IBM, DataDirect Networks, Hitachi and Copan Systems disk storage; and IBM and Sun StorageTek tape drives.
The SDSC isn't waiting until the new data center is finished to save on energy and costs, however. As routine IT equipment upgrades occur, Thornton and his 18-person department look for some basic features that can help improve energy efficiency, such as power management options, multicore chips and high-density servers, such as blades, that make the most of their capacity.
"When we work with vendors, we really hold them to keep their energy consumption down," he says.
Higher use, lower cost
Thornton describes a method of achieving energy efficiency in the data center that, while simple, seems to run contrary to the way data centers have been structured for years: By getting the maximum use out of existing servers, companies can avoid powering unused portions of their machines and therefore spend less money on energy. This approach -- known as high-utilization and often dependent on such technologies as virtualization, which can turn dedicated servers into multifunction computers -- means servers spend the most time possible processing tasks and the least time sitting idle, yet still on.
"What people aren't paying attention to is the idea of high utilization," says Andrew Kutz, an analyst with Burton Group. Organizations -- in particular, online ones where delays in server response times can lead directly to dollars lost -- have become so obsessed with making sure their servers are functioning at top efficiency during peak traffic times that they're not using a large percentage of the servers' processing power during off times -- yet they're still powering these servers.
Let's say an application server can handle a maximum of x simultaneous requests from users, yet that level of requests is achieved only a small percentage of the time. That means the rest of the time, a portion of the server's processing power is unused but still draws energy.
"Companies need to focus on their servers being highly utilized," Kutz adds. That can be achieved without affecting performance through such techniques as virtualization and power management.
For example, the SDSC is planning to replace about 20 dual, single-core-processor Dell servers. Each consumes about 400 watts with five dual, quad-core processor Dell servers that each consume about 300W. Thornton plans to virtualize the operating system and combine applications on these fewer servers so there is minimal impact to users.
What this consolidation means to energy efficiency -- and the bottom line -- is significant. Each of the existing 20 servers consumes 400W, with a total power draw of 8,000W; the five new servers each will consume 300W, with a total power draw of 1,500W, but will offer the same computing power.
"During less than their lifetime, [the new servers] will have paid for themselves on utilities savings alone," Thornton says. "Not to mention savings of freed-up power and cooling equipment, space, and other data center infrastructure."
In addition, the SDSC has instituted a chargeback mechanism for users who install their own IT equipment in the center, Thornton says, charging them more for high-consumption machines and less for energy-efficient architectures.
"We're trying to encourage the users of the data center to limit power use and upgrade what they have on the floor to new machines," Thornton says. "The key is aligning incentives with targeted outcomes -- in this case, energy efficiency."