For Most Data Centers, Liquid and Air Cooling Will Not be Mutually Exclusive

#CPEXPO | CHANNELPARTNERSCONFERENCE.COM#DATACENTERWORLD | DATACENTERWORLD.COM
For Most Data Centers,
Liquid and Air Cooling
Will Not be Mutually Exclusive
Lars Strong, Senior Engineer
Upsite Technologies

#DATACENTERWORLD | DATACENTERWORLD.COM
Data Center World – Certified Vendor Neutral
Each presenter is required to certify that their presentation will
be vendor-neutral.
As an attendee you have a right to enforce this policy of
having no sales pitch within a session by alerting the speaker
if you feel the session is not being presented in a vendor
neutral fashion. If the issue continues to be a problem, please
alert Data Center World staff after the session is complete.

Speaker Background
• Thought leader and recognized expert on data center
optimization
• Currently serves as the Senior Engineer and Company
Science Officer of Upsite Technologies
• Certified U.S. Department of Energy Data Center Energy
Practitioner (DCEP) HVAC Specialist
• Previous AFCOM Presentations:
• How IT Decisions Impact Facilities: The Benefit of Mutual Understanding
• Designing, Deploying, and Managing Efficient Data Centers
• Myths of Data Center Containment
• Understanding the Science Behind Data Center Airflow Management
• Data Center Cooling Efficiency: Understanding the Science of the 4 Delta T’s

Adoption of Liquid Cooling
~1970-1995
• Mainframes
1990-2000
• Gaming
• Custom
built PCs
2010-2015
• HPC driven
liquid
cooling in
the data
center
2005-2010
• Chilled
door
2015-
• Direct
contact
• Full
immersion

• Sandia National Laboratory is completing a purpose built
(primarily) liquid cooled data center in New Mexico
• 14,000 sq. ft.
• 10MW
• 85% warm water liquid cooled
• Augmented air cooling of 1.5MW (15%)
Sample Case Study

• Density
• HPC – latency
Drivers for Liquid Cooling

• Density
• HPC – latency
• Performance
• Higher clock speeds (research, modeling, finance)

• Density
• HPC – latency
• Performance
• Economy
• Efficiency
• IT fans represent ~20% of IT load. Affects all aspects of power
distribution (switch gear, UPS, PDU, etc.)
• Warm water cooling can produce significant CapEx and OpEx savings

• Density
• HPC – latency
• Performance
• Economy
• Efficiency
• IT fans represent ~20% of IT load. Affects all aspects of power
distribution (switch gear, UPS, PDU, etc.)
• Warm water cooling can produce significant CapEx and OpEx savings
• More accurate PUE
• Server fans are on the IT load side of the PUE equation when they are
actually part of the cooling infrastructure

• Air is “understood,” familiar
• Care and handling paradigm shift – immersion
• White space – immersion
• Risk – liquid in cabinet, electrocution, outages
• No standards or many options
• Capital cost
• Complexity of running two systems
• Resistance due to hydrophobia
Resistance to Liquid Cooling

• “Power in heat out always.” – Ken Brill
• Every kW of electricity becomes a kW of heat that needs to be
removed from the building
• kW of IT consumption
• kW of fan motor consumption
• kW of transformer losses
• kW of distribution losses (resistance)
Definition of Liquid Cooling

• “Power in heat out always.” – Ken Brill
• Every kW of electricity becomes a kW of heat that needs to be
removed from the building
• kW of IT consumption
• kW of fan motor consumption
• kW of transformer losses
• kW of distribution losses (resistance)
• Liquid Cooling: Removal of heat by a liquid
• When the first loop that takes the heat away from the IT components is
liquid, then it is liquid cooling
• Many loops are often required to remove heat from the building
Definition of Liquid Cooling

Proximity of Liquid to the Heat Source
Mainframe
Server
In Row
Rear Door
Touch
Immersion

• Rear door cooling
• Active (fans), or passive
• In row cooling
• Over row cooling
• In the top of cabinet cooling
Types of “Liquid” Cooling (Close Coupled Air Cooling)

• Touch Cooling
• Direct Liquid Cooling (DLC) to components, “plumbing”
Types of Liquid Cooling
CoolIT Lenovo
AsetekChilldyne

• Touch Cooling
• Fixed cold plate, heat spreaders conduct heat to
cold plate
Aquila

• Immersion
• Single Phase: using ElectroSafe, a formulated dielectric oil
• Green revolution
• Pump energy to keep moving
• 2 Phase: using dielectric fluids (Novec, Fluorinert)
• ICEOTOPE, LiquidCool

Liquid Cooling
Close Coupled Air Cooling
Immersion Touch
Immersion
Direct Liquid
Cooling
Fixed Cold Plate Rear Door In Row
% of Heat
Captured by
Liquid
100% ~65% to 75% ~93% 0% N/A
% of Heat
Captured by Air
0% ~25% to 35% ~7% 100% N/A
Effectiveness of Various Types of Liquid Cooling
Cabinet Level

Air Touch Immersion
Compute/Processing
Y Y Y
Switching (lots of cords/ports) Y Difficult / In development Difficult / In development
Solid state storage Y Y Y
Spinning media Y Y
Spinning media if sealed
(HDD)
Power supplies Y In development Y
What Does Air and Liquid Cool Well and Not Cool Well

• Space available for air and water are different
• Liquid is often running through small tubing
• 1/4” – 0.05 sq. in.
• 1/8” – 0.012 sq. in.
• Air is likely to have several square inches 3.0 sq. in. (60x – 1/4” tube)
• Density of air ~0.074 lbs. / ft3 (1 lbs. – 12.4 ft3)
• Density of water ~ 62.3 lbs. / ft3 (1 lbs. – 0.016 ft3) (842x better
than air)
• Specific heat of air ~993 J/kg °C
• Specific heat of water ~4200 J/kg °C (only 4x better than air)
Physics of Liquid Cooling

• Convection heat transfer from a surface to an fluid is governed
by the equation: q = hA(Tw - Tf)
• h = the convection heat transfer coefficient
• A = the surface area
• Tw = the temperature of the surface
• Tf = the temperature of the fluid
• Coefficient of heat transfer for moderate speed air is~100 and
moderate flow water is ~ 3000
• Water is 30x better than air

• Convection heat transfer from a surface to an fluid is governed
by the equation: q = hA(Tw - Tf)
• h = the convection heat transfer coefficient
• A = the surface area
• Tw = the temperature of the surface
• Tf = the temperature of the fluid
• Coefficient of heat transfer for moderate speed air is~100 and
moderate flow water is ~ 3000
• Water is 30x better than air
• Surface area available
• Water flowing over chip - 1 sq. in.
• Air flowing over heat sink – 10 sq. in.
• Taking surface area into consideration water is 3x better than
air

• Liquid cooling is the removal of heat by a liquid
• Depending on the component and solution, only a portion of the
heat can be removed by a liquid
• Not all equipment will be liquid cooled
• The question is not which is better, but how can you use both
liquid and air effectively?
• Liquid has always been used in the data center, just how close
• Liquid and air cooling will not be mutually exclusive
Key Takeaways

Questions & Answers
Lars Strong, P.E.
Senior Engineer, Upsite Technologies
lds@upsite.com
Follow Upsite for the latest news and information on data center airflow optimization.
Facebook InstagramLinkedInTwitter YouTube Upsite Blog

For Most Data Centers, Liquid and Air Cooling Will Not be Mutually Exclusive

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to For Most Data Centers, Liquid and Air Cooling Will Not be Mutually Exclusive

Similar to For Most Data Centers, Liquid and Air Cooling Will Not be Mutually Exclusive (20)

Recently uploaded

Recently uploaded (20)

For Most Data Centers, Liquid and Air Cooling Will Not be Mutually Exclusive

Editor's Notes