According to data from the industry association Bitkom from 2023, the performance capacities of German data centers have almost doubled in the past twelve years - from 1131 MW in 2010 to 2341 MW in 2022. A much steeper increase in performance is imminent. However, the increase in power densities will have an even more immediate impact, particularly due to high-performance computing and the rapid spread of AI algorithms. This is because a lot of computing power is needed to train them, which new types of processors (GPUs) can provide in a very small space. This also means a radical change in cooling technology. From around 30 kW power per rack, air is no longer enough. Then only liquid cooling can help, preferably directly on the processor. As a side effect, so to speak, this also serves sustainability: the effectiveness of liquid-cooled solutions is many times higher than that of air-cooled solutions because water has a much better heat storage and conductivity capacity.

At the European conference of the Open Compute Project (OCP) in June, Rittal presented a Coolant Distribution Unit (CDU) in OCP format for the first time. In the Open Compute Project, hyperscalers, OEMs and various trades in the data center industry work collaboratively on highly standardized IT infrastructure.

The CDU uses a water mixture for Single Phase Direct Liquid Cooling, direct chip cooling with a single-phase (non-evaporating) cooling liquid. Two CDU variants are provided in the 'Open Rack V3' format (21 inch), the development of which was driven forward by Rittal at OCP: one for installation directly in the racks to be cooled, one for in-row installation with over 1 MW cooling capacity for bayed racks. Liquid-to-air variants that do not require a facility water connection are planned, including a 19-inch version with full integration into Rittal's 'RiMatrix' system platform.

Quick access thanks to liquid-to-air version

The redundantly designed Coolant Circulation Units and other modules are inserted in the rack like server plug-in units and can be easily replaced during operation.

© Rittal

Hyperscalers and operators of large data centers in particular are likely to use the liquid-to-liquid solution in large numbers and thus set standards. However, there are still many technological and structural issues to be resolved for the data center as an overall system. This requires extensive testing.

However, the agile colocation industry cannot wait for this. As the operator, colocators provide the data center infrastructure and various companies place their own IT systems there. Most of them want to offer their customers good conditions for AI and HPC as quickly as possible. This is where the liquid-to-air versions come into play, which cool the processors with water, but then release the heat into the air via the rear door of the rack or a side cooler. Although these solutions do not achieve the cooling performance and efficiency of liquid-to-liquid solutions, they can be used more quickly in data centers without a water connection. With them, Colocator can carry out its own tests with less effort and investment or create individual 'HPC islands' (High Performance Computing) for its customers in air-cooled data centers. These versions thus act as 'levers' to bring Direct Liquid Cooling into data centers as an enabling technology for AI.

Increasing opportunities for waste heat utilization

Up to 1500 liters of water mixture rush through the system every minute to remove the heat of 1 MW.

© Rittal

With liquid-to-liquid solutions, on the other hand, the heat can be better transferred to the building's primary water circuit or to waste heat users, also due to significantly higher waste heat temperatures. This improves the efficiency values of the data center even more and enables compliance with regulations, as the use of waste heat in data centers will soon be prescribed by the Energy Efficiency Act. Rittal sees this as one of the reasons why the proportion of such solutions will increase in the medium term, even though they require major changes to the entire infrastructure.

The in-rack solution with water connection is available for cooling capacities of 30, 60 and 100 kW and a temperature spread of 8 Kelvin. At 1.5 bar, the system achieves a flow rate of 45, 90 or 150 l/min. The pumps have a redundant design and the coolant distribution is located at the bottom of the rack. Power is supplied via the standardized 'DC busbar' in the rack. The modular controller and an optional battery-operated UPS are located at the top. The cooling liquid used is water enriched with 25 % polyglycol, a proven mixture in industry and vehicle cooling systems. The liquid distribution for the supply and return flow is built into the rear of the rack. Here, Rittal has taken the OCP approach of standardized power distribution in the rack a step further with a manifold for the water circuit and drip-free connectors in the rack. The pluggable quick couplings can be equipped with pressure sensors that can be replaced during operation. Temperature sensors can also be integrated. The CDU is connected to the building's water system via heat exchangers.

Simple service

In the in-row version, up to five coolant circulation units (CCUs), each with three pumps, are located above the plate heat exchanger. Together, they provide a flow rate of up to 1500 l/min. With several small pumps, the output can be controlled more precisely than with just a few larger pumps, which in turn benefits energy efficiency. The cooling efficiency of the CDU is just over 1.

As it already achieves its maximum output with four plug-in units, one plug-in unit can be removed at a time without interrupting operation. Any defective pumps can be easily replaced. Each plug-in unit has its own connection to the busbar and to the separate control unit.

All sensors, for example for temperature, pressure and conductivity, can be replaced in the in-row solution without interrupting operation. The quick couplings only lose a few drops of liquid during coupling, which are compensated for via the liquid reservoir. This is done automatically, as is the venting.

The author: Michael Nicolai is Head of Rittal IT Sales Germany in Herborn.

© Rittal

The consistent separation of the secondary circuit on the rack side and the primary circuit on the building side ensures the necessary quality of the cooling water in the rack, which flows through the fine capillaries of the heat sinks on the processors. The quality of the local tap water supplier is usually sufficient for the water in the primary circuit on the building side.