Liquid Cooling for AI Data Centers: Direct-to-Chip vs Immersion, CDUs, Rack Density and PUE Math for GPU Clusters (2026)

Liquid Cooling for AI Data Centers: Direct-to-Chip vs Immersion, CDUs, Rack Density and PUE Math for GPU Clusters (2026)

Liquid Cooling for AI Data Centers: Direct-to-Chip vs Immersion, CDUs, Rack Density and PUE Math for GPU Clusters (2026)

A single NVIDIA GB200 NVL72 rack is specified at roughly 120 kW. Uptime Institute’s 2025 survey found that few facilities exceed 30 kW per rack. That gap of four to one is the whole story of ai data center liquid cooling in 2026. Air cannot carry that much heat through a 600 mm wide cabinet at any fan speed an operator would tolerate, so water has moved from the facility perimeter into the server tray itself. The shift is no longer optional. NVIDIA’s rack-scale systems are designed around liquid cooling, and in 2026 the company publicly discussed coolant entering the chip at up to 45 °C. That figure changes chiller economics, water use and heat reuse all at once. Yet most coverage stops at “liquid is more efficient.” This analysis goes further. It works through the heat-load and flow math for a 130 kW rack, explains what a coolant distribution unit actually does, and compares direct-to-chip and immersion on real constraints. It also shows why PUE can mislead you during the transition.

What this covers: rack density trends from GB200 to Rubin Ultra, the TCS and FWS loop architecture, ASHRAE W-classes, worked Q = ṁ·cp·ΔT and pipe sizing, CDU design and redundancy, direct-to-chip vs single-phase and two-phase immersion, PFAS, a PUE and WUE comparison, retrofit vs greenfield, and the failure modes that take liquid-cooled halls down.

Context and Background

For two decades, data center cooling meant moving air. Computer room air handlers pushed cold air under a raised floor or down a contained aisle, servers pulled it across heat sinks with internal fans, and chillers rejected the heat outside. That model scaled comfortably to 10 to 15 kW per rack and, with hot-aisle containment and rear-door heat exchangers, to perhaps 30 to 40 kW. Uptime Institute’s 15th annual Global Data Center Survey (2025) reported that average rack densities continue to rise, with more racks in the 10 to 30 kW range, but that “few facilities exceed 30 kW” and extreme densities remain rare. The same survey noted that average PUE had barely moved for the sixth consecutive year.

AI training and inference clusters broke that envelope in one product cycle. Accelerators went from roughly 700 W per GPU to 1,000 W and then 1,200 to 1,400 W per package in the Blackwell generation. NVIDIA put 72 of them, plus 36 Grace CPUs and 18 NVLink switch trays, into one rack so that the NVLink domain could stay on copper. Power density followed. NVIDIA’s own sustainability blog states a full-rack density of 120 kW for GB200 and GB300 NVL72, and NVIDIA contributed the GB200 NVL72 rack and its liquid cooling manifolds to the Open Compute Project (OCP) in October 2024.

Liquid cooling itself is not new. IBM mainframes used water decades ago, and Google says it first deployed liquid cooling with TPU v3 in 2018. Google now reports running liquid cooling “at GigaWatt scale across more than 2000 TPU Pods” with fleet-wide CDU availability of about 99.999% since 2020. What is new is that liquid is now the default for merchant GPU racks sold to colocation providers and enterprises, not just for a few hyperscalers with in-house mechanical teams.

The power side of this problem is covered in our analysis of 800 V DC power architecture for AI data centers. Every watt delivered to a GPU becomes a watt of heat the cooling plant must remove. For the rack itself, see our breakdown of the NVIDIA GB300 NVL72 Blackwell Ultra architecture. The external reference most engineers start from is the OCP Advanced Cooling Facilities reference design guidance, which describes the loop architecture used in this post.

The Reference Architecture for AI Data Center Liquid Cooling

AI data center liquid cooling moves chip heat into a closed coolant loop, the technology cooling system (TCS), through cold plates or immersion tanks. A coolant distribution unit (CDU) transfers that heat across a heat exchanger into the facility water system (FWS), which rejects it through dry coolers, cooling towers, chillers or a heat-reuse network.

AI data center liquid cooling architecture from GPU cold plates through rack manifold and CDU to facility water, dry coolers, towers, chillers and heat reuse
Figure 1: The two-loop heat path in a liquid-cooled AI hall. The TCS stays clean and small; the FWS is large, warm and connected to the outdoors.

Figure 1 reads left to right as the heat travels. Cold plates bolted to each GPU and CPU package pass heat into a water-glycol coolant. Rack manifolds collect flow from each compute tray through dripless quick disconnects. The CDU pumps that coolant, filters it, and moves its heat across a plate heat exchanger into facility water. The facility loop then has several possible sinks: a dry cooler that uses only fans, an evaporative cooling tower, a chiller used only on the hottest days, or a heat-reuse connection such as a district heating network.

Two loops, because chips and buildings want different water

The single most important design choice is keeping two hydraulically separate loops. The TCS touches micro-channel cold plates with fins measured in fractions of a millimetre. It needs tightly controlled chemistry, fine filtration and a fluid that will not corrode copper or grow biofilm. OCP’s PG25 guidelines, published in October 2026, describe the reference TCS fluid as inhibited propylene glycol at 24.5 to 27.5% by volume. They recommend sidestream filtration for systems over 250 gallons that turns the full volume over at least once every 24 hours.

The FWS is the opposite. It is large, may run through a cooling tower open to the atmosphere, and carries whatever chemistry the building’s water treatment allows. Connecting cold plates directly to a tower loop would clog them within months. The CDU’s heat exchanger is the firewall between the two. That separation costs a few degrees of temperature, called the approach, but it buys reliability. It is the same reason car engines use a closed coolant loop and a radiator, rather than pumping lake water through the block.

The OCP guidance notes one alternative: a control valve with a recirculating pump can blend facility water to the right temperature without a heat exchanger. It explicitly warns that this provides no loop isolation and has a limited flow range. For GPU racks worth several million dollars each, isolation usually wins.

Warm water changes the economics, not just the efficiency

ASHRAE TC 9.9’s fifth-edition thermal guidelines renamed the liquid cooling classes after their upper facility water supply temperature: W17, W27, W32, W40, W45 and W+ (formerly W1 to W5, with W40 newly added). The edition also tightened what compliance means. Equipment rated to a W-class must run at full, unthrottled performance across that class’s whole range. Before, some vendors labelled hardware by a range it merely tolerated.

Why does this matter? Every degree of warmer supply water widens the number of hours per year when the outdoor air alone can cool the loop. NVIDIA’s June 2026 liquid cooling blog describes coolant entering the chip at up to 45 °C and leaving at roughly 55 °C. It uses a 75% water and 25% propylene glycol mix, and NVIDIA’s Ali Heydari is quoted saying chillers are needed “outside of maybe 1% of the year” in some climates. The same post says raising chiller plant temperature by one degree can cut cooling energy costs by about 4%. Treat that as a rule of thumb from the vendor, not a law of physics, but the direction is right.

A 45 °C supply also produces a 55 °C return. The OCP facilities guidance says heat reuse becomes practical above about 50 °C. Warm-water cooling therefore turns the waste stream into a product, which we return to in the PUE section.

Rack density is now a facility parameter, not an IT parameter

The third principle is that rack power density for AI is set by the interconnect, not by floor space. NVLink’s copper reach is what forces 72 GPUs into one rack. Dropping density means splitting the NVLink domain, which costs performance. Our post on NVLink vs InfiniBand vs RoCE for GPU cluster networks explains why that scale-up domain matters so much for collective operations.

The roadmap makes this worse. NVIDIA showed the Kyber rack at GTC in March 2025, with Jensen Huang saying “each rack is 600kW” for Rubin Ultra NVL576, planned for the second half of 2027. Google has publicly said ML will require more than 500 kW per IT rack before 2030, and it has proposed ±400 VDC power delivery for racks up to 1 MW. Treat the 600 kW and 1 MW figures as announced plans, not shipping products. Even so, mechanical plants built in 2026 will outlive several GPU generations. A hall designed for 120 kW racks with no headroom will be stranded quickly.

Deeper Analysis: Heat-Load Math, CDUs and Cooling Topologies

The physics of liquid cooling fits on one line: Q = ṁ · cp · ΔT. Heat removed (Q, in kW) equals mass flow (ṁ, in kg/s) times specific heat (cp, in kJ/kg·K) times the temperature rise of the coolant (ΔT, in K). Everything else (pipe sizes, pump power, CDU capacity, ride-through time) follows from that equation and from fluid properties.

Worked example: a 130 kW rack (illustrative)

Take an illustrative rack with 130 kW of IT load. That is slightly above NVIDIA’s published 120 kW figure, to leave margin. The numbers below are teaching values, not a vendor specification. Real designs must use the rack vendor’s flow and pressure-drop tables.

Assume the liquid loop captures 100% of the heat and the coolant rises 10 K across the rack (for example, 45 °C in, 55 °C out, matching NVIDIA’s described temperatures). For pure water, cp is about 4.18 kJ/kg·K. For PG25, it is lower, roughly 3.9 kJ/kg·K (approximate; check your fluid’s datasheet).

Fluid ΔT Mass flow ṁ = Q / (cp·ΔT) Approx. volume flow
Water 10 K 130 / (4.18 × 10) = 3.11 kg/s ≈ 187 L/min
PG25 10 K 130 / (3.9 × 10) = 3.33 kg/s ≈ 196 L/min (ρ ≈ 1.02 kg/L)
PG25 7 K 130 / (3.9 × 7) = 4.76 kg/s ≈ 280 L/min
PG25 15 K 130 / (3.9 × 15) = 2.22 kg/s ≈ 131 L/min

Illustrative values computed from Q = ṁ·cp·ΔT; fluid properties are approximate.

Two lessons fall out. First, glycol costs about 5% more flow than water for the same heat, because its specific heat is lower. That is the price of freeze protection and biostatic properties. Second, ΔT is the strongest lever. Narrowing ΔT from 10 K to 7 K raises flow by 43%. Pump power then rises much faster, because pressure drop grows roughly with the square of flow in turbulent regimes. Widening ΔT reduces flow but raises chip temperatures at the outlet end of a cold-plate chain. That is why vendors specify both a maximum inlet temperature and a minimum flow per tray.

Now compare air. Air has a density near 1.2 kg/m³ and cp near 1.005 kJ/kg·K. Moving 130 kW with a generous 15 K air temperature rise requires 130 / (1.2 × 1.005 × 15) ≈ 7.2 m³/s, or about 15,000 CFM through one cabinet. Per unit volume, water carries about 3,500 times more heat per kelvin than air (4.18 MJ/m³·K vs about 0.0012 MJ/m³·K). NVIDIA’s blog rounds the density comparison to “nearly 1,000x.” Either way, air at this density is a fan-noise and fan-power problem with no good answer.

Pipe sizing intuition (illustrative)

Velocity limits set pipe diameter. The OCP facilities guidance recommends 3 to 5 ft/s (about 0.9 to 1.5 m/s) for mild steel and below 3 ft/s for copper, to limit erosion and noise. At 196 L/min (3.27 L/s) and 1.5 m/s, the required cross-section is 0.00327 / 1.5 ≈ 0.0022 m². That gives an internal diameter of about 53 mm for a single rack. Keep copper below 0.9 m/s and the diameter grows to about 68 mm.

Scale to a row of ten such racks, 1.3 MW, and the header carries about 33 L/s. At 1.5 m/s that needs an internal diameter of roughly 167 mm, which means DN150 to DN200 steel or stainless. A 10 MW hall needs facility headers in the DN400-plus range. This is why retrofits so often fail on routing rather than on chiller capacity. There is simply no room under an existing raised floor, or over the aisle, for pipes of that size alongside cable trays and busway.

Pump power and ride-through (illustrative)

Hydraulic pump power is P = flow × pressure rise / efficiency. Assume the row loop needs 250 kPa total head, through cold plates, quick disconnects, manifolds, the CDU heat exchanger and filters, and pumps run at 60% wire-to-water efficiency. Then 0.033 m³/s × 250 kPa / 0.6 ≈ 13.8 kW. That is about 1% of the 1.3 MW the row removes. These are illustrative numbers, but the order of magnitude is why liquid cooling’s transport energy is small compared with chilled-air systems. Most of the savings come from eliminating server fans and compressor hours.

Ride-through matters more than steady-state efficiency. Suppose each rack’s share of the TCS holds 200 L of PG25 (illustrative), and the GPUs can tolerate a 10 K coolant rise before throttling. The stored thermal buffer is 200 L × 1.02 kg/L × 3.9 kJ/kg·K × 10 K ≈ 8 MJ. At 130 kW, that lasts about 60 seconds. Compare that with a typical air-cooled room, where the air mass and raised floor give minutes of ride-through at low density. At high density they give almost nothing. The practical conclusion is that CDU pumps must be on UPS. A generator-start gap of 10 to 15 seconds is survivable only if circulation never stops. Google’s description of Project Deschutes explicitly puts CDUs on UPS with redundant pump and heat-exchanger units.

Inside a coolant distribution unit

A coolant distribution unit is a pump skid with a heat exchanger, and the OCP glossary defines it exactly that way: “heat exchanger and pumps that isolate cooling loops.” A production CDU adds a lot around that core:

  • Pumps in N+1 or 2N, usually variable speed, with automatic changeover on loss of differential pressure.
  • A brazed or gasketed plate heat exchanger, sized for the approach temperature. CoolIT’s CHx2000, cited by NVIDIA, is rated at 2 MW with a 5 °C approach. Google’s Project Deschutes CDU, contributed to OCP in 2025, is also a 2 MW class unit.
  • Filtration sized to the finest passage in the system. OCP’s PG25 guideline asks fluid makers to filter to 50 microns or finer, with 25 microns preferred.
  • An expansion tank and pressurisation, to absorb thermal expansion and keep pump suction pressure positive.
  • Sensors and controls: supply and return temperature, differential pressure, flow, conductivity or glycol concentration, leak detection, and a controller that regulates TCS supply temperature by modulating a facility-side valve.

CDUs come in two families. Liquid-to-liquid (L2L) units reject heat into facility water and scale to megawatts. Liquid-to-air (L2A) units, often rack-sized “sidecars,” reject TCS heat into room air through a coil and fans. L2A lets you run liquid-cooled racks in a hall with no facility water. The trade-off is capacity, since each sidecar handles tens of kilowatts to low hundreds. It also dumps heat back into the room that the existing air plant must still remove. L2A is a bridge, not a destination.

Placement is the other axis. OCP distinguishes in-rack, in-row and facility-level CDUs. In-row CDUs are the common choice for GPU halls because they keep the TCS volume small per failure domain. A facility-level CDU serving an entire hall is cheaper per kilowatt but creates a huge blast radius if its loop is contaminated.

Direct-to-chip vs immersion

Comparison of direct-to-chip cooling, single-phase immersion and two-phase immersion heat paths into the facility water loop
Figure 2: Three ways to get GPU heat into facility water. Direct-to-chip cools the hottest packages; immersion submerges the whole board.

Direct-to-chip cooling (also called direct liquid cooling, DLC) bolts a cold plate onto each high-power package. Coolant flows through micro-channels a short distance from the silicon, giving very high heat transfer coefficients exactly where the heat flux is highest. Lower-power components such as memory, NICs, power stages and drives may still be air cooled. That leaves a residual air load: perhaps 10 to 25% in mixed designs (illustrative). Newer trays push that toward zero. Glenn Lockwood’s notes on Blackwell Ultra, citing NVIDIA material, describe the GB300 compute tray as designed to be 100% liquid cooled, with a liquid manifold replacing the row of fans. DLC is what NVIDIA, Google and the OCP rack ecosystem have standardised on. It also keeps the server serviceable in a normal rack with normal tools.

Single-phase immersion submerges entire servers in a tank of dielectric fluid, typically a synthetic hydrocarbon or mineral oil. A pump circulates the fluid through a heat exchanger, which is effectively a CDU, to facility water. Every component is cooled, there are no fans at all, and the tank is silent. The weaknesses are physical. Hydrocarbon dielectrics have roughly a quarter of water’s thermal conductivity and about half its specific heat (approximate properties). Natural or low-velocity forced convection around a 1,000-plus watt package therefore struggles without special heat sinks. Servicing means lifting dripping boards out of a tank, and fluid compatibility with cables, labels and thermal interface materials has to be qualified part by part.

Two-phase immersion uses a fluid that boils at a low temperature on the chip surface. Vapour rises, condenses on a coil at the top of the tank, and drips back. Latent heat of vaporisation makes this extremely effective at high heat flux. The problem has been the fluids. The incumbents were fluorinated, and 3M announced in December 2022 that it would exit all PFAS manufacturing by the end of 2025, which included its Novec and Fluorinert lines. DatacenterDynamics reported that two-phase vendors, including LiquidStack, Zutacore and Wiwynn, referenced Novec. Single-phase immersion vendors using hydrocarbons were not expected to be affected. Chemours announced an HFO-based alternative, Opteon 2P50, which it says it does not consider PFAS. The article also notes there is no single global PFAS definition, so regulatory risk remains a live diligence item.

The immersion cooling vs direct liquid cooling decision in 2026 therefore turns less on thermal performance than on ecosystem. GPU vendors ship rack-scale systems designed, validated and warrantied for cold plates. Immersion can work, and it has real advantages for edge sites, harsh environments and lower-density mixed racks. But for a GB200 or GB300 class deployment, choosing immersion means taking on qualification and support risk the OEM has not taken on. Always get written confirmation of warranty coverage before submerging a system that was designed for cold plates.

Criterion Direct-to-chip (single-phase) Single-phase immersion Two-phase immersion
Heat captured to liquid High; residual air on some designs ~100% ~100%
Peak heat flux handling Very high via micro-channels Moderate; needs enhanced sinks Very high via boiling
GPU OEM rack support (2026) Mainstream for NVL72-class Limited, case by case Niche; check support
Serviceability Normal rack, dripless QDs Lift boards from tank Sealed tank, vapour management
Fluid risk Water-glycol leaks near electronics Fluid cost, compatibility PFAS regulation, fluid supply
Floor loading Rack weight only Heavy tanks of fluid Heavy tanks plus sealing
Best fit Hyperscale and colo GPU halls Edge, crypto, mixed density Specialist high-flux sites

Qualitative comparison; verify OEM support and fluid status for your specific hardware.

PUE and WUE Math: Why the Metric Can Lie During the Transition

Power usage effectiveness (PUE) is total facility energy divided by IT equipment energy. It was introduced in 2006, promoted by The Green Grid from 2007, and standardised as ISO/IEC 30134-2 in 2016. Water usage effectiveness (WUE), also from The Green Grid (2011) and later ISO/IEC 30134-9 (2022), is annual site water use divided by IT energy, in litres per kWh.

The trap is that server fans count as IT load. An air-cooled GPU server spends a meaningful share of its power on fans. Liquid cooling removes most of that fan power from the denominator. Total energy falls, but PUE improves by less than it should, and in a badly instrumented comparison it can even appear to get worse.

A worked 10 MW comparison (illustrative)

The numbers below are illustrative. They show the mechanism and are not measured data from any facility.

Line item (MW) Air-cooled, chilled water Direct-to-chip, W45 dry coolers
Useful compute (chips, memory, network) 9.20 9.20
Server fans, counted as IT 0.80 0.15
IT load (PUE denominator) 10.00 9.35
Chillers, towers, CRAH fans 3.00 0.00
Dry cooler fans and CDU pumps 0.00 0.50
Residual air handling included above 0.30
Electrical losses (UPS, transformers) 0.80 0.75
Lighting and other 0.20 0.20
Total facility 14.00 11.10
PUE 1.40 1.19
Total energy per MW of useful compute 1.52 1.21

Illustrative model. Fan share, plant power and losses vary widely by site, climate and hardware.

The reported PUE drops from 1.40 to 1.19, about 15%. But total energy per unit of useful compute drops from 1.52 to 1.21, about 20%. The extra five points hide in the IT denominator, where fan savings are invisible to PUE. Researchers recognised this years ago and proposed ITUE and TUE (IT-power usage effectiveness and total-power usage effectiveness), which push the boundary inside the server to separate fans and power supplies from compute. If your board or sustainability team will judge the liquid cooling investment on PUE alone, show them both rows.

The reverse trap also exists. A facility that adds liquid-cooled GPU racks to an under-loaded hall will see IT load rise sharply while fixed overheads stay flat. PUE will improve dramatically even if the cooling plant is no more efficient. The Wikipedia summary of PUE criticism notes the general point that PUE is “practically meaningless unless the IT is working at full capacity.”

Water: from evaporation to closed loops

Water is where warm-water liquid cooling changes the picture most. NVIDIA’s blog cites conventional cooling tower systems at roughly 2.6 million gallons per megawatt per year. Convert that: 2.6 million gallons is about 9.8 million litres, and one megawatt for a year is 8.76 million kWh. That is about 1.1 L/kWh of WUE if the load runs flat out. A dry-cooler loop at 40 to 45 °C supply evaporates nothing in normal operation, so its site WUE approaches zero, apart from any adiabatic assist used on the hottest days. NVIDIA’s “up to a 100% reduction in water use” claim rests on exactly that mechanism.

The trade-off is energy. Dry coolers need larger coils and more fan power than towers at the same approach, and they lose capacity on hot days. In a hot, dry climate, a designer may choose towers for energy efficiency, accepting high WUE. Alternatively they may choose dry coolers plus chiller trim, accepting a higher peak electrical demand. There is no universal answer. Run hourly weather data through both designs before committing, and remember that water stress is local while electricity carbon intensity depends on the grid.

Heat reuse

A 55 °C return stream is warm enough to feed some district heating networks directly, or to serve as an excellent source for a heat pump that lifts it to 70 to 80 °C. Our explainer on how heat pumps work and what limits their efficiency shows why a small temperature lift gives a high coefficient of performance. Reused heat does not lower PUE, because PUE only sees energy into the facility. Energy reuse metrics such as The Green Grid’s ERE (energy reuse effectiveness) and ERF (energy reuse factor, ISO/IEC 30134-6) were created to capture it. If heat reuse is part of your business case, report ERF alongside PUE.

Operations: Failure Modes and Control Sequences

Sequence of CDU controller, standby pump, BMS, rack power shelf and job scheduler responding to a flow drop or leak in a liquid-cooled AI rack
Figure 3: A pump-failure or leak response sequence. The cooling system and the job scheduler must be integrated, because the thermal buffer lasts roughly a minute.

Figure 3 shows the sequence that should exist on day one and that many first deployments lack. A flow sensor or leak detector trips. The CDU controller starts the standby pump and raises an alarm to the building management system (BMS) and data center infrastructure management (DCIM) platform, tagged with the loop ID. The BMS tells the cluster scheduler which racks are at risk, so jobs can checkpoint before thermal throttling corrupts a training step’s timing. If temperatures still rise, the controller throttles or isolates the rack power shelf. Only after valves are confirmed closed and the loop is stable does the alarm clear.

The integration between facility controls and the workload scheduler is the most neglected part of ai data center liquid cooling. A 60-second ride-through is long enough for automated action and far too short for a human to read a ticket.

Trade-offs, Gotchas, and What Goes Wrong

Leaks and quick disconnects. Every tray connection is a potential leak point, and an NVL72 rack has dozens of them. Dripless quick disconnects, including the OCP Universal Quick Disconnect (UQD) form factor, minimise spillage on mating. However, damaged O-rings, mis-seated couplings and side loads from heavy hoses still cause weeps. Use leak detection rope under manifolds and drip trays at the rack base, and pressure-decay test every tray after service. Some vendors offer negative-pressure TCS designs that pull coolant through the loop under vacuum, so that a breach draws air in rather than pushing liquid out. They trade leak containment for lower available pressure head.

Condensation. The OCP guidance cites ASHRAE TC 9.9’s recommended maximum dew point of 15 °C. Liquid supplied above 15 °C carries minimal condensation risk. Warm-water designs are naturally safe here, but chilled-water retrofits that feed the TCS at 12 to 14 °C to “help” a struggling rack can sweat inside the server. Always control TCS supply above room dew point, with a margin.

Water chemistry. Inhibitors deplete, glycol concentration drifts after top-ups with plain water, and dissimilar metals corrode galvanically. OCP’s PG25 guideline lists fresh-fluid pH, hardness below 25 ppm, chloride below 5 ppm and sulfate below 10 ppm. It warns that sulfur-cured EPDM seals can leach sulfate into the fluid. Monthly sampling and a written fluid-maintenance contract are cheaper than one fouled cold plate. Flush thoroughly before commissioning; construction debris is the most common cause of early filter blockage.

Pump and CDU redundancy. A single CDU per row is a single point of failure for 1 MW or more of GPUs. Specify N+1 pumps at minimum, put pumps and controllers on UPS, and consider dual CDUs with cross-connected headers for training clusters, where one rack’s loss can stall a synchronous job across thousands of GPUs.

Air in the loop and control instability. Entrained air causes pump cavitation, noisy flow readings and hot spots in cold plates. Poorly tuned temperature control can oscillate when the facility valve and CDU pump loops fight each other. Commission with deliberate load steps, as OCP’s guidance does at 100%, 125% and 150% of design, rather than only at steady state.

Floor loading and logistics. Liquid-cooled racks are heavy, and filled immersion tanks are heavier. Older raised floors may need reinforcement. Integrating and leak-testing racks at the factory, a practice OCP addresses in its liquid cooling integration and logistics work, avoids filling hundreds of trays on site.

Decision tree for choosing air, liquid-to-air CDU, liquid-to-liquid CDU, retrofit, greenfield warm-water or immersion based on rack power density for AI
Figure 4: A first-pass decision tree by rack density and site. The thresholds are illustrative planning bands, not standards.

Retrofit vs Greenfield

Figure 4 frames the decision most operators actually face. The density bands are illustrative. Below about 40 kW, a well-contained air hall with rear-door heat exchangers often remains the cheapest path. Between roughly 40 and 100 kW, the question becomes whether facility water is available. If not, L2A sidecars buy time. If yes, L2L CDUs on the existing chilled-water or condenser-water plant are the standard retrofit. Above 100 kW, which is GB200 and GB300 NVL72 territory, you are choosing between a heavy retrofit and a purpose-built warm-water hall.

Factor Retrofit existing air hall Greenfield warm-water hall
Time to first rack Months, if piping routes exist 18 to 36 months including power
Facility water temperature Constrained by existing chillers, often W17 to W27 Designed for W40 or W45
Chiller hours Many; chillers remain primary Few; trim only in hot climates
WUE Depends on existing towers Near zero with dry coolers
Pipe routing Hardest constraint; ceiling or under-floor congestion Designed in from the start
Floor loading Must verify; may need reinforcement Slab on grade, designed for weight
Heat reuse Rarely practical at low return temps Feasible at 50 to 55 °C return
Stranding risk at 600 kW racks High Moderate if headers are oversized

Qualitative planning comparison; durations are indicative ranges, not quotes.

Retrofits look cheaper on a spreadsheet and are often slower in reality, because the binding constraint is physical space for DN150 to DN400 pipes. They also leave you running warm-water-capable GPUs on cold chilled water, which throws away the main efficiency gain. Greenfield sites face a different bottleneck: grid interconnection, discussed in our analysis of behind-the-meter power and nuclear PPAs for AI data centers. The capital side of these choices, and why hyperscalers are building rather than retrofitting, is covered in our hyperscaler AI capex and compute economics analysis.

Designing for the next generation

If you build in 2026, size for at least the next generation. NVIDIA said at CES 2026 that its Rubin platform was in full production, with Vera Rubin NVL72 as the rack-scale system. Rubin Ultra in the Kyber rack is announced for the second half of 2027 at around 600 kW. Exact Vera Rubin rack power has been widely reported by third parties but, at the time of writing, we could not confirm a definitive NVIDIA-published rack figure; treat third-party numbers as estimates.

What you can design for regardless of the exact figure: oversized facility headers, valved stubs for future CDUs, structural capacity for heavier racks, warm-water-capable heat rejection, and a power distribution path compatible with higher-voltage DC. The mechanical plant should be the part of the hall you never have to rip out.

Practical Recommendations

Start from the heat path, not from the product catalogue. Decide your facility water class first, because it sets chiller hours, water use and heat-reuse potential for twenty years. Then choose CDUs and topology to fit the GPU vendor’s validated design, rather than asking the vendor to fit your legacy chilled water.

Treat cooling as part of the cluster’s control plane. The roughly one-minute thermal buffer in a high-density rack means facility alarms must reach the job scheduler automatically. Treat fluid chemistry as a maintenance discipline with its own budget and owner, not a commissioning task. Finally, report energy honestly: PUE alongside total energy per unit of compute, WUE, and energy reuse where it applies.

A short checklist before signing a liquid cooling design:

  • Confirm the GPU vendor’s required TCS supply temperature, flow per tray, pressure drop and fluid. Get the W-class in writing.
  • Run Q = ṁ·cp·ΔT for your worst-case rack and size headers at 1.5 m/s maximum velocity, with at least one generation of headroom.
  • Specify N+1 CDU pumps on UPS and calculate thermal ride-through from loop volume.
  • Integrate leak and flow alarms with BMS, DCIM and the cluster scheduler, then test the sequence with real load steps.
  • Write a PG25 or equivalent fluid-maintenance plan: filtration, sampling frequency and approved inhibitor top-ups.
  • Model PUE, total energy per compute and WUE with hourly weather data for both tower and dry-cooler options.
  • For immersion, obtain written OEM warranty coverage and confirm the fluid’s PFAS and regulatory status.

Frequently Asked Questions

Why do AI data centers need liquid cooling instead of air?

Because rack power has outrun what air can carry. NVIDIA specifies GB200 and GB300 NVL72 racks at about 120 kW, while Uptime Institute’s 2025 survey found few facilities exceed 30 kW per rack. Removing 130 kW with air at a 15 K rise needs roughly 15,000 CFM through one cabinet (illustrative calculation). Water carries thousands of times more heat per unit volume, so cold plates can remove the same heat with modest flow, small pipes and far less fan power. Dense NVLink domains make spreading GPUs across more racks costly.

What is the difference between direct-to-chip and immersion cooling?

Direct-to-chip cooling attaches cold plates to the hottest packages, such as GPUs and CPUs, and pumps water-glycol through them. Other components may still use some air. Immersion submerges the whole server in a dielectric fluid, either a hydrocarbon that stays liquid (single-phase) or a fluid that boils on the chip (two-phase). Immersion captures nearly all heat and removes fans, but OEM support for rack-scale GPU systems centres on cold plates. Servicing tanks is also harder, and two-phase fluids carry PFAS regulatory risk.

What does a coolant distribution unit do?

A coolant distribution unit isolates the clean technology cooling loop that touches the servers from the facility water loop. It contains pumps, usually redundant, a heat exchanger, filtration, an expansion tank, and sensors and controls that hold the coolant supply temperature and flow steady. Liquid-to-liquid CDUs reject heat into facility water and scale to megawatts; 2 MW units are now offered. Liquid-to-air CDUs reject heat into room air and suit retrofits without facility water, at limited capacity.

What are ASHRAE W-classes for liquid cooling?

They are facility water temperature classes defined by ASHRAE TC 9.9. The fifth edition names them after the upper supply temperature limit: W17, W27, W32, W40, W45 and W+, which replaced the older W1 to W5 names and added W40. Equipment compliant with a class must run fully unthrottled across that class’s range. Higher classes allow more hours of chiller-free cooling with dry coolers, reduce water use, and produce return water warm enough for heat reuse.

Does liquid cooling improve PUE?

Usually yes, but PUE understates the gain. Server fans count as IT load, so removing them shrinks the PUE denominator while also cutting total energy. In an illustrative 10 MW model, PUE fell from 1.40 to 1.19, about 15%, while total energy per unit of useful compute fell about 20%. Compare total energy per compute and WUE, not PUE alone. Also beware the reverse effect: filling an under-loaded hall with dense GPU racks improves PUE even if the cooling plant is unchanged.

Is two-phase immersion cooling still viable after 3M’s PFAS exit?

It is viable but carries supply and regulatory risk. 3M announced in December 2022 that it would exit PFAS manufacturing, including the Novec and Fluorinert lines used by several two-phase vendors, by the end of 2025. Alternatives exist. Chemours’ HFO-based Opteon 2P50 is one, and Chemours says it does not consider the fluid PFAS. Because PFAS definitions differ across jurisdictions, check current regulations where you operate. Fluid availability and end-of-life handling belong in the contract.

Further Reading

References

  1. NVIDIA blog on Blackwell platform water efficiency and liquid cooling: GB200/GB300 NVL72 full-rack density of 120 kW. https://blogs.nvidia.com/blog/blackwell-platform-water-efficiency-liquid-cooling-data-centers/
  2. NVIDIA, liquid cooling for AI factories blog (June 2026): 45 °C inlet, ~55 °C outlet, 75/25 water and propylene glycol, chiller hours, 2.6 million gallons per MW per year for towers. https://blogs.nvidia.com/blog/liquid-cooling-ai-factories/
  3. eeNews Europe, “Nvidia offers 120kW liquid-cooled Blackwell rack as industry standard” (October 2024): GB200 NVL72 OCP contribution. https://www.eenewseurope.com/en/nvidia-offers-120kw-liquid-cooled-blackwell-rack-as-industry-standard
  4. DatacenterDynamics, “Nvidia’s Rubin Ultra NVL576 rack expected to be 600kW, coming second half of 2027.” https://datacenterdynamics.com/en/news/nvidias-rubin-ultra-nvl576-rack-expected-to-be-600kw-coming-second-half-of-2027
  5. Open Compute Project, Advanced Cooling Facilities Reference Design Guidance white paper. https://www.opencompute.org/documents/ocp-acf-reference-design-guidance-white-paper-r1-pdf/
  6. Open Compute Project, PG25 Guidelines V1.0.0 (October 2026). https://www.opencompute.org/wp-content/uploads/2026/10/OCP-PG25-Guidelines-V1.0.0.pdf
  7. Upsite Technologies, “Major changes to ASHRAE’s fifth edition of Thermal Guidelines, part 3: liquid cooling chapter updates” (summary of ASHRAE TC 9.9 W-classes). https://www.upsite.com/blog/major-changes-to-ashraes-fifth-edition-of-thermal-guidelines-part-3-liquid-cooling-chapter-updates/
  8. Google Cloud, “Enabling 1 MW IT racks and liquid cooling at OCP EMEA Summit” (2025). https://cloud.google.com/blog/topics/systems/enabling-1-mw-it-racks-and-liquid-cooling-at-ocp-emea-summit
  9. Uptime Institute, Global Data Center Survey 2025 (press release). https://www.businesswire.com/news/home/20250730149641/en
  10. 3M, “3M to Exit PFAS Manufacturing by the End of 2025” (December 20, 2022). https://news.3m.com/2022-12-20-3M-to-Exit-PFAS-Manufacturing-by-the-End-of-2025
  11. DatacenterDynamics, “Two-phase cooling will be hit by EPA rules and 3M’s exit from PFAS forever chemicals.” https://www.datacenterdynamics.com/en/news/two-phase-cooling-will-be-hit-by-epa-rules-and-3ms-exit-from-pfas-forever-chemicals
  12. DatacenterDynamics, “Chemours launches two-phase immersion cooling fluid.” https://datacenterdynamics.com/en/news/chemours-launches-two-phase-immersion-cooling-fluid
  13. ISO/IEC 30134-2 (PUE) and ISO/IEC 30134-9 (WUE), Information technology — Data centres — Key performance indicators; background via https://en.wikipedia.org/wiki/Power_usage_effectiveness and https://en.wikipedia.org/wiki/Water_usage_effectiveness

By Riju — about

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *