I am old enough to remember the 1960’s when IBM Mainframes used de-ionized water delivered by micro-bore pipes to cool the CPUs. (In fact I remember a spillage during a mainframe move that resulted in every single auto spares shop in south east England being raided for deionized water).
In a recent statement IBM claim that direct water could also be the cooling technology of the future. IBM say that removing excess heat from data centers is as much as 4000 times more efficient via water than it is by air. This revitalized cooling technology, was reintroduced in 2005, when IBM launched the Cool Blue water-cooled products.
There are two basic approaches that get cold water closer to the case or heat-sinks of the hottest components (generally the CPUs) in the first the IBM technology uses cold plates physically bolted to the CPUs with centrally delivered chilled water channeled through them. This removes heat efficiently but is mainly targeted at getting the case temperature down so that the Power 6 chips can be over-clocked reliably delivering more performance for the same basic silicon.
The second approach uses cold water delivery to coils in a rear door (Emerson deliver a similar product). The IBM’s Rear Door Heat eXchanger is 4 inches thick and weighs in at a hefty 70 lbs. IBM claim that the door can absorb as much as 50% of the heat coming from the server rack.
The novice would say that water and electronics don’t mix well but experienced data center managers know that water is the main medium that is used in Computer Room Air Conditioners (CRAC) units to cool the raised floor area. Bringing water closer to the CPU core will improve the efficiency of heat transfer by as much as 4000 times. So in summary if we get the liquid right up against the CPUs we need to pump 4000 times less water than air to get the same value of cooling. IBM’s Cool Blue doors are significantly more efficient than the CRAC units in the same room as they are closer to the source of heat but crucially they have a negative effect on the CRAC units they are designed to supplement.
CRAC units operate most efficiently when the return airflow is as hot as possible, Cool Blue doors cool the hot airflow and therefore reduce the efficiency of the rest of the room. Shame that the doors were not fitted to the front of the cabinet where they could ensure uniformly cold air being drawn into the front of each server.
“We’ll see a return to liquid-cooling in most IT solutions,” said Doug Neilson, a consultant in IBM’s systems and technology group, at the IDC Enterprise Data Centre conference in London. The technique, once used in specialised supercomputers and large mainframes, could now have more general use as companies try to become more energy efficient, he said.
“Water-cooling is a better way to recycle the heat energy,” said Neilson. “If you cool by air it is much harder to capture and condense it.” To harness that energy, it must be transferred to liquid, so it is more efficient to use liquid cooling directly, he said.
I really agree with IBM and am certain that liquid cooling has such a significant advantage over air cooling that the economics and density advantages will be overwhelming. Handling water in a data center is business as usual and we must not be frightened of it. The equipment vendors need to get more joined up with the data center M&E guys and start looking at delivering cooling at the chip level for all high performance, high density systems.
The advantages could be overwhelming:
- No refrigeration needed, even in the hottest climates free air cooling could be used as input liquid temperatures of 130 degrees F (50 degrees C) could be used
- Waste heat could be delivered at useful temperatures like 160 degrees F (65 degrees C)
- Pump energy could be minimized enabling PUE levels of 1.05 or better to be achieved
- Data Centers could be silent as there would be no need for fans
- New equipment could be thermally neutral, not adding any extra heat load to the site
- No humidity problem, no humidifiers
- No need for a raised floor
- Massive improvement in reliability due to thermal stability
Liquid cooling falls into the joined up thinking that The Hot Aisle promotes. Well done IBM.