I met with Dr. Bruno Michel who is IBM’s, Zurich based, advanced thermal packaging manager yesterday. We spoke about IBM’s view of cooling technology in the 5 to 10 year horizon and what I heard was interesting and credible. IBM have been working with liquid cooling for some time, in the HydroCube, in the new Z11 mainframe when it is launched (probably next year) and now some new chip based technology.

Here is the basic physics lesson:

Large Scale Electronics are made up of transistors and other active and passive components. Current flows through the circuits constructed on the silicon substrate and flows across transistor junctions. As these transistors are switching very fast, they use high currents and the junctions get hot. The solid state substrate conducts heat to the chip case and then onto a heatsink made of copper or aluminium.

So we need to get air passed over the fins of the heatsink to allow convection cooling to occur. This is done by chilling water, passing it through coils in a CRAC (Computer Room Air Conditioning) unit inside the data center and propelling it under fan assisted power towards these heatsinks.

So we have cold water, cooling air that then cools the metal heatsink that then cools the chip case and this cools the transistor junctions. All of these steps need a temperature gradient to work so the cold water needs to be colder that the cool air coming out of the CRACs and the transistor junctions are hotter than the chip case. Actually we have a temperature drop in the order of 67° Celsius as the chilled water tends to be at 18° C and the transistor junctions at 85° C. This temperature difference is called the Delta T (ΔT) of the cooling system. Air is an appallingly bad conductor of heat and hence the need for a large ΔT.

• Air = 0.0245 W/(mK) Thermal conductivity
• H2O = 0.6 W/(mK) Thermal conductivity

Water is 3300 times more efficient at carrying heat than air. So we need 3300 m³ of air to perform the same cooling function as 1 m³ of water!

If we can get the water physically closer to the transistor junctions the ΔT can be smaller and the efficiency of the cooling system can be improved. The closest that the water could get is to be channeled through the chip substrate in micro bore channels etched into the silicon. Bruno’s team are working on just such a design. Boris is working on chip scale liquid cooling. This promises a highly efficient ΔT of 20° C or better and the possibility of exhaust water being delivered at 80° C.

Water at 80° C is useful for heating, desalination and hydroponics. This offers the promise of a PUE (Power Usage Efficiency) of less than 1 as some of the compute load energy can be reused for another purpose.

Getting liquid closer to the transistor junctions is the key to extreme high density and Boris’s team is looking at the possibility of creating layers inside the silicon. This means that the distance between components drops and signalling speeds can increase. The numbers of connections can also go up and as a result functionality and performance can be build in at the chip level rather than at the motherboard level.

Unfortunately all of this is some time away – perhaps as much as ten years, but the good news is that although this approach is patented, the chip manufacturers have a policy of sharing intellectual property and so we may see some of this in other vendor’s offerings, not just IBM’s Power series.