Der Weg vom Transistor zum emissionsfreien Rechenzentrum · Der Weg vom Transistor zum emissionsfreien Rechenzentrum Dr. Bruno Michel Manager Advanced Thermal Packaging IBM Zurich

© 2010 IBM Corporation

Der Weg vom Transistor zum emissionsfreien Rechenzentrum Dr. Bruno Michel Manager Advanced Thermal PackagingIBM Zurich Research LaboratorySäumerstrasse 4, CH-8803 Rüschlikon, [email protected]

IBM Research, Zurich Research Laboratory, Rüschlikon, Switzerland


Aquasar Inauguration, May 6, 2010

2

Green Datacenter Market Drivers and Trends Increased green consciousness, and rising cost of power

IT demand outpaces technology improvements – Server energy use doubled 2003-2008;

temporary slowdown due to economic crisis resume of power growth is not sustainable

– Koomey Study: Server use 1.2% of U.S. energy

ICT industries consume 2% world wide energy– Carbon dioxide emission like global aviation

Real Actions Needed

Brouillard, APC, 2006

Future datacenters dominated by energy cost; half energy spent on coolingSource IDC, 2009



3

From the Individual Transistor to the Globe

100

kmDa

y

5’00

0 km

Year

40’0

00 k

m10

0 ye

ars

Morteratsch Glacier, Switzerland, 1985

Morteratsch Glacier, Switzerland, 2007

Glacier recession 1985-2007: 400 meters!

Pow

er

Stat

ion

Cont

inen

tal

Pow

erNe

ts

Glo

bal

Tem

pera

ture

- Thermal issues propagate up to the world climate- Global length- and decade time-scales involved - A holistic view is required to solve these problems

IT to become part of the solution to climate challenge!



4

Zero-Emission Data Centers

High-performance chip-level cooling improves energy efficiency AND reduces carbon emission:– Cool chip with ΔT = 20ºC instead of 75ºC– Save chiller energy: Cool datacenter with

T > 60ºC hot water – Re-use: Heat 700 homes with 10 MW datacenter

Need for carbon footprint reduction – EU, IPCC, Stern report targets– Chillers use ~50% of datacenter energy – Space heating ~30% of carbon footprint

Zero-emission concept valuable in all climates– Cold and moderate climates:

energy savings and energy re-use – Hot climates: Free cooling, desalination

Europe: 5000 district heating systems– Distribute 6% of total thermal demand– Thermal energy from datacenters absorbed



5

First Prototype at IBM Rüschlikon

Reduce cooling energy by water cooling– Cooling the chip with “hot” water (60°C)– Free cooling: no energy-intensive chillers needed

Reuse waste heat for remote heating

– Re-uses 75% of blade energy

– Recyclable heat (60°C) for remote heating

Prototype– Similar Power for air and

60°C liquid cooled version– Large fan power reduction– Basis for “Aquasar” system

Direct attached micro-channel cold plate

Experimental validation: Air vs. cold, vs. hot water cooling



6

Target Reach world record in performance (MFlops/W)

and low emission (MFlop/gCO2) Reduce energy cost by a factor two Lead standardization for future datacenters PUEreuse less than 1

Technical Details 33 QS22 and 9 HS22 IBM

BladeCenter® Servers Two chassis liquid cooled and one air cooled Infiniband communication equipment and

a storage server Closed cooling loop with 20 liters of water The coolant flow in rack is 30 liters per minute

Connection to ETHheating system

Aquasar Rack

Aquasar Hot-Water Cooled HPC Cluster for ETHLiquid-cooled Blade Center Chassis



7

Water Cooled IBM BladeCenter® Server

© IBM Research 2010



8

Scalable Heat Removal by 3D Interlayer Cooling

“Aquasar” - a milestone towards future more efficient computers 3D integration requires interlayer cooling for stacked logic chips Bonding scheme to isolate electrical interconnects from coolant

Through silicon via electrical bonding and water insulation scheme

A large fraction of energy in computers is spent for data transport Transport energy depends on distance: Shrinking computers

saves energy Interaction with ETH on CMOSAIC NanoTera project Vision for more compact and more efficient exascale computers

with “hot” water cooling like Aquasar

cross-section through fluid port and cavities

Test vehicle with fluid manifold and connection



9

Aquasar Key Message and Outlook Reduce emission up to 85% through heat re-use Save up to 40% of energy and substantially reduce energy Performance evaluation and optimization as part of CCEM project

Roadmap for large efficiency increase in 15 years – Interlayer cooling of 3D chips in CMOSAIC NanoTera Project

Scale up to full size data centers pending

IT driven solutions for the climate challenge



10

IBM Team and Business PartnersCore Team for Aquasar Build: IBM Research - Zurich: Ingmar Meijer, Stephan Paredes, Thomas Brunschwiler, Patrick

Ruch, Marcus Oestreicher, Thorsten Kramp, IBM Germany, Böblingen: Jürgen Marschall, Manfred Ries,

Walter Weber, Martin Bachmaier, Gottfried Goldrian IBM Schweiz: Bruno Battaglia

Acknowledgements: Peter Buhler, Marcel Bürge, Ralph Heller, Daniel Jehle,

Markus Kirschner, Urs Kloter, Michael Malms, Oliver Rettig, Walter Riess, Daniel Ronzani, Sandra Rufener, Hansruedi Steinauer, Erich Rütsche, Jörg Schanze, Gerhard Schoor, Jürgen Stumpp, Heiner Tschopp,Kurt Wasser, and Martin Witzig

Business Partners: Wolverine Tube, Inc. Walter Meier AG APC by Schneider Electric

Documents

Der Weg vom Transistor zum emissionsfreien Rechenzentrum · Der Weg vom Transistor zum emissionsfreien Rechenzentrum Dr. Bruno Michel Manager Advanced Thermal Packaging IBM Zurich