cat /infra/reality.txt
A neutral, order-of-magnitude reference for sanity-checking an AI facility claim in 60 seconds—from accelerator nameplate power to rack density, PUE, water, construction cost, and grid schedule.
volatile hardware and market figures are dated inline
These are comparison anchors, not universal constants. “Derived” rows show transparent arithmetic from cited inputs; vendor power is nameplate unless the row says otherwise.
| Number | Unit / object | What it means | Scope | Primary source |
|---|---|---|---|---|
| 700 W | NVIDIA H200 SXM | Maximum configurable TDP for one SXM accelerator; it is not whole-server power. | Jul 2026 | NVIDIA |
| 1,400 W | AMD MI355X OAM | Typical board power for one flagship liquid-cooled-capable accelerator module. | Jul 2026 | AMD |
| 10.2 kW | DGX H100/H200 server | Maximum AC input for an 8U, eight-GPU system—about 1.275 kW per installed GPU including CPUs, memory, fabric, and losses. | current guide | NVIDIA |
| 40.8 kW | 4× DGX H100 rack | Recommended maximum server demand for four air-cooled DGX H100 systems in one rack. | current guide | NVIDIA |
| 72 GPUs | GB200 NVL72 rack | One liquid-cooled rack-scale NVLink domain also carries 36 Grace CPUs. | Jul 2026 | NVIDIA |
| ≈120 kW | GB200 NVL72 rack | Full-load rack power including 18 compute nodes and rack components—not only the 72 GPUs. | Mission Control 2.1 | NVIDIA |
| 7 MW | 10,000 H200 SXM GPUs | Accelerator-only TDP: 10,000 × 700 W. Servers, network, and cooling raise the connection requirement. | derived | NVIDIA input |
| 14 MW | 10,000 MI355X GPUs | Accelerator-only TBP: 10,000 × 1.4 kW; do not quote this as facility load. | derived | AMD input |
| 8.19 MW | 10K H200 facility floor | 7 MW accelerator TDP × Microsoft FY25 PUE 1.17. Still excludes the non-GPU parts already missing from 7 MW. | derived floor | Microsoft PUE |
| 1.56 PUE | industry average | Uptime’s 2024 survey average: 100 MW of IT corresponds to 156 MW total facility power. | 2024, n=526 | Uptime |
| 1.17 PUE | Microsoft fleet | Owned-fleet global average; an efficient hyperscaler reference, not a guarantee for a new site. | FY25 | Microsoft |
| 0.27 L/kWh | Microsoft WUE | Annual cooling and humidification water per IT kWh across Microsoft’s owned fleet. | FY25 | Microsoft |
| ≈171K gal/day | 100 MW IT at WUE 0.27 | 100,000 kW × 24 h × 0.27 L/kWh ÷ 3.785 L/US gal. Site water only; power-plant water is separate. | derived | Microsoft input |
| ≈228K gal/day | 100 MW IT at WUE 0.36 | Same calculation using LBNL’s modeled 2023 U.S. fleet average just over 0.36 L/kWh. | 2023 model | LBNL |
| 18–27°C | server inlet air | ASHRAE recommended dry-bulb envelope for air-cooled A1–A4 equipment; allowable limits are broader by class. | ASHRAE guidance | ASHRAE |
| 100 kW+ | liquid-cooled rack | Density class OCP says hyperscalers were considering; GB200’s ≈120 kW is a shipping example. | 2024 reference | OCP |
| 500 kW+ | near-future AI rack | OCP’s July 2026 infrastructure-standardization callout; a planning horizon, not today’s fleet average. | Jul 2026 | OCP |
| 1 kWh → 1 kWh heat | IT energy conversion | Nearly all server electricity becomes heat in the room; “efficient compute” does not remove the heat-rejection obligation. | physics | LBNL |
| 70–80% | potentially recoverable heat | LBNL’s cited technical potential; usefulness still depends on temperature, distance, and a coincident heat customer. | 2025 review | LBNL |
| 60–80% | AI server operating power | LBNL’s modeled range as a fraction of rated power—evidence against equating nameplate with annual average draw. | 2024 model | LBNL |
| 80% / 40% | training / inference time | LBNL modeled AI server operational time—not measured utilization for every deployment. | 2024 model | LBNL |
| 415 TWh | global data centers | About 1.5% of world electricity consumption in 2024. | 2024 estimate | IEA |
| ≈945 TWh | global data centers | IEA base-case demand in 2030—more than double 2024, but explicitly scenario-dependent. | 2030 forecast | IEA |
| 649 TWh | U.S. data centers | LBNL 2026 reference case for 2030; compounded uncertainty spans 521–843 TWh. | Jun 2026 model | LBNL |
| $9.5–13.3/W | U.S. construction | 2025 traditional data-center build benchmarks: Charlotte to Silicon Valley. Facility construction scope—not land, IT hardware, finance, or energy. | 2025 USD | T&T |
| +7–10% | AI liquid-cooling premium | Turner & Townsend’s U.S. average premium versus equivalent air-cooled capacity. | 2025 analysis | T&T |
| 2–3 years | data center to operation | IEA’s general facility lead-time anchor; transmission, generation, permitting, and equipment can take longer. | 2025 report | IEA |
Enter a cluster assumption. The output makes the accounting boundary explicit; it does not size redundancy, storage, networking, switchgear, or phasing.
| Metric | Precise meaning | Concrete example | Gotcha |
|---|---|---|---|
| PUE | Total annual facility energy ÷ annual IT equipment energy. Ideal is 1.0. | 156 MWh ÷ 100 MWh = 1.56 | It does not measure useful compute, water, carbon, or embodied energy; climate and boundary choices matter. |
| WUE | Annual site water for cooling/humidification ÷ annual IT energy, in L/kWh. | 27,000 L ÷ 100,000 kWh = 0.27 L/kWh | On-site WUE excludes water consumed to generate electricity unless separately reported as source WUE. |
| ERE | Energy Reuse Effectiveness credits energy reused outside the data center against total facility energy. | (156 − 30) MWh ÷ 100 MWh = 1.26 | A heat-recovery nameplate is not reused energy; count metered delivery to a real sink. |
| CUE | Annual operational carbon emissions ÷ annual IT energy, usually kg CO₂e/kWh IT. | 40,000 kg ÷ 100,000 kWh = 0.40 kg/kWh | State whether emissions are location- or market-based and whether Scope 3 is excluded. |
| Rack density | Maximum or measured IT power in one rack, expressed in kW/rack. | 4 × 10.2 kW DGX = 40.8 kW/rack | An “average 30 kW rack” says nothing about a 120 kW hot spot the cooling loop must survive. |
| IT load | Power delivered to compute, storage, and network equipment at the defined measurement point. | 100 MW IT × PUE 1.17 = 117 MW facility | Vendor GPU TDP is not IT load; it omits the rest of the server and network. |
| Critical load | Protected load the electrical design must sustain through the agreed redundancy path. | 80 MW critical at N+1 needs spare path capacity | It is a design/resilience boundary, not automatically the same as average IT draw. |
| Nameplate | Rated maximum or design limit used for safe electrical and thermal provisioning. | 10.2 kW max DGX ≠ 10.2 kW annual average | Using nameplate as measured demand overstates energy; using average demand to size breakers is unsafe. |
The bar uses a logarithmic scale so 700 W and 1 GW fit in one frame. Cost envelopes apply the 2025 U.S. $9.5–13.3/W facility-construction range only where a facility rung makes sense; they exclude IT hardware and land.
Residential analogy uses EIA’s 2022 average purchase of 10,791 kWh/customer/year = 1.232 kW continuous. It compares energy rate, not reliability, coincident peak, transmission, or customer count served.
Start with measured or engineered whole-server power. Add network and storage to reach IT load, then multiply by PUE for facility energy. For a quick upper envelope, four 10.2 kW DGX H100 systems make a 40.8 kW server rack; at PUE 1.17, that rack’s allocated facility burden is 47.7 kW before shared-load nuances.
facility power = (servers + network + storage) × PUE
Nameplate sizes safe circuits. Coincident peak demand sizes the service and capacity contract. Annual energy determines much of the bill and footprint. LBNL’s 2024 model used 60–80% of rated power for operating AI servers and 80% / 40% operational time for training / inference.
For 10.2 kW nameplate, 70% operating power × 80% operational time gives a 5.71 kW annual-average model—not permission to provision only 5.71 kW.
Contain hot and cold aisles, seal bypass paths, manage blanking panels, and hold server inlets in the vendor/ASHRAE envelope. NVIDIA’s four-DGX rack is 40.8 kW and explicitly caps the reference pattern at four systems for thermodynamic reasons.
Use when: the equipment and rack density remain inside a validated airflow design. Do not use: a fleet-average kW/rack to excuse a hotter local rack.
Cold plates take high-flux heat from CPUs/GPUs to a coolant distribution unit; residual DIMM, PSU, and network heat may still need air. AMD specifies 2.1 L/min per MI355X OAM and coolant inlet up to 43°C for its eight-GPU platform.
Use when: rack power and chip heat flux outrun economical air delivery. Do not assume: “liquid cooled” means zero facility water consumption.
Immersion places electronics in dielectric fluid; rear-door heat exchangers intercept exhaust at the rack. Both change service procedures, materials compatibility, leakage risk, floor load, and the facility loop.
Decision: compare total solution density—including CDUs and service clearance—not the bare tank or rack number.
| Cooling claim | Sanity check | Decision implication |
|---|---|---|
| “Air handles it” | Show inlet temperature, airflow, pressure, containment, fan power, and the worst rack—not only average density. | 40.8 kW is a published air-cooled rack design; ≈120 kW GB200 ships liquid cooled. There is no universal cutoff. |
| “Warm water is free cooling” | Show facility-water supply/return temperatures and outdoor design conditions. | Warmer loops can expand economizer hours and heat reuse, but a CDU/pump still consumes power. |
| “Liquid cooling saves water” | Separate closed-loop coolant volume from cooling-tower evaporation and source-water use. | A closed chip loop can reject heat through a wet tower or a dry cooler; WUE reveals the site result. |
| “We can reuse the heat” | Name the customer, distance, required temperature, seasonal load, and metered MWh delivered. | LBNL cites 70–80% recoverable potential; low-grade heat without a sink has zero reuse value. |
Microsoft defines WUE as annual liters used for cooling and humidification divided by annual IT kWh. Its FY25 owned-fleet values range from 0.03 L/kWh in EMEA to 0.34 in the Americas, with 0.27 global. Climate, cooling design, and load make a single global comparison hazardous.
100 MW IT × 24 h × 0.27 L/kWh = 648,000 L/day = 171,202 US gal/day
Site WUE excludes water consumed by the electricity supply. LBNL models both because thermoelectric generation and hydropower can add a location-dependent indirect footprint. “Zero water for cooling” can be accurate at the facility and incomplete for the workload.
Decision: report site withdrawal, site consumption, and source-energy water separately; do not add unlike boundaries.
Turner & Townsend’s 2025 traditional build benchmarks span $9.5/W in Charlotte to $13.3/W in Silicon Valley. A 100 MW facility therefore implies about $950M–$1.33B in comparable construction scope; high-density liquid cooling adds a reported 7–10% U.S. premium.
Excluded: land, utility off-site work, finance, owner costs outside methodology, and the accelerators/servers themselves. A vendor quote that combines these categories cannot be compared directly.
IEA says a data center can be operational in two to three years, while the broader energy system needs longer. ERCOT’s 2026 process studies ≥75 MW large loads in batches and was tracking 438 GW of requests; that queue is requests, not forecast realized demand.
PJM says its reformed generation queue targets one-to-two-year agreements, but large-load service also depends on utility studies, substations, transmission, transformers, permits, and construction. Do not substitute a generation-queue statistic for a load energization promise.
| Gate | Evidence to request | Common schedule failure |
|---|---|---|
| Site + permits | Controlled parcel, zoning path, environmental/water permits, construction permit milestones. | “Shovel-ready” means marketing-ready, not permit-complete. |
| Utility service | Signed study/service agreements, MW by year, firm vs interruptible terms, required upgrades. | Treating an application or queue position as reserved capacity. |
| Long-lead equipment | Purchase orders and delivery dates for transformers, switchgear, generators, chillers/CDUs. | Building shell finishes before the power train arrives. |
| Commissioning | Integrated systems test, load-bank plan, staged rack acceptance, operational runbooks. | Calling utility energization “production ready.” |
| Compute delivery | Allocation, network fabric, storage, spares, firmware/software validation. | Power exists while usable accelerator capacity does not. |
A 100 MW continuous campus consumes 2.4 GWh/day. A 100 MW generator nameplate cannot cover that load if it is intermittent, derated by temperature/altitude, down for maintenance, or constrained by fuel. Specify dependable capacity, ramp, startup, minimum load, storage duration, black-start behavior, emissions permits, and N+1 contingencies.
required generator nameplate ≈ continuous load ÷ capacity factor — before reserves
They are related but different processes. ERCOT’s Batch Zero is for large users ≥75 MW; PJM’s quoted one-to-two-year agreement target is for generation interconnection. Ask the serving utility/ISO for the actual study path, milestones, upgrade scope, and energization conditions for the site.
Training tends toward long, synchronized runs with high accelerator occupancy and network intensity. Inference can be latency-sensitive, bursty, geographically distributed, and provisioned for peak demand. LBNL’s model uses 80% operational time for training and 40% for inference, but a real fleet must be measured at server, rack, and service boundaries.
Checkpointing, workload migration, power capping, and staged commissioning can lower coincident demand or accelerate partial energization. Quantify how many MW can move, for how long, with what notice, and at what recovery cost. “Interruptible” without a tested workload contract is not firm grid flexibility.
Use the dated source, not this page’s cached number, when approving a design or investment.
Print-and-pin reference · W/kW/MW = power; kWh/TWh = energy · source abbreviations below
| Number | Object | Meaning / boundary | Source |
|---|---|---|---|
| 700 W | H200 SXM | max configurable GPU TDP | NV |
| 1,400 W | MI355X | one OAM typical board power | AMD |
| 10.2 kW | DGX H100/H200 | 8U eight-GPU server maximum | NV |
| 40.8 kW | 4× DGX rack | recommended air-cooled server demand | NV |
| 72 GPUs | GB200 NVL72 | one liquid-cooled NVLink domain | NV |
| ≈120 kW | GB200 NVL72 | full rack at full load | NV |
| 7 MW | 10K H200 | GPU-only TDP; derived | NV |
| 14 MW | 10K MI355X | GPU-only TBP; derived | AMD |
| 8.19 MW | 10K H200 floor | GPU-only 7 MW × PUE 1.17; derived | NV/MS |
| 1.56 | industry PUE | 2024 average; 100 MW IT → 156 MW facility | UI |
| 1.17 | Microsoft PUE | FY25 owned-fleet global average | MS |
| 0.27 L/kWh | Microsoft WUE | FY25 site water per IT energy | MS |
| 171K gal/day | 100 MW IT | at WUE 0.27; derived | MS |
| 228K gal/day | 100 MW IT | at WUE 0.36; derived | LBL |
| 18–27°C | server inlet | ASHRAE recommended A1–A4 air | ASH |
| 100 kW+ | liquid rack | density class cited by OCP | OCP |
| 500 kW+ | future AI rack | OCP Jul 2026 planning horizon | OCP |
| 1:1 | energy → heat | nearly all IT electricity becomes heat | LBL |
| 70–80% | heat recovery | technical potential, not delivered reuse | LBL |
| 60–80% | operating power | LBNL model fraction of AI rated power | LBL |
| 80% / 40% | training / inference | LBNL modeled operational time | LBL |
| 415 TWh | global 2024 | data-center electricity, ≈1.5% | IEA |
| 945 TWh | global 2030 | IEA base case | IEA |
| 649 TWh | U.S. 2030 | LBNL reference; 521–843 uncertainty | LBL |
| $9.5–13.3/W | U.S. build | 2025 traditional facility construction | T&T |
| +7–10% | liquid premium | U.S. high-density vs air-cooled | T&T |
| 2–3 years | facility | general build-to-operation anchor | IEA |
NV nvidia.com/docs.nvidia.com · AMD amd.com · UI Uptime Institute 2024 · MS Microsoft FY25 · LBL Lawrence Berkeley National Laboratory 2024/2026 · ASH ASHRAE · OCP Open Compute Project · IEA Energy and AI 2025 · T&T Data Centre Construction Cost Index 2025. Full links and boundary notes: cheatsheets.davidveksler.com/ai-infrastructure-numbers.html