What are AI data centers, actually?

Sitting at the kitchen table, you type a simple question into your phone: "I'm looking for a coffee machine under 200 euros that is easy to clean." Moments later, the first words appear on screen. Your phone did not compute that answer by itself.

Behind the word "cloud" sits a physical chain of fiber-optic cables, routers, specialized servers, power grids and cooling systems. Our coffee-machine prompt is a fictional teaching example: we trace the journey of that message from your screen into the server hall and back.

As soon as you tap Send, the app packages your prompt together with the conversation history into an encrypted request. Across mobile or Wi-Fi networks and through fiber-optic links, the packet reaches a data center.

There, a front-end gateway coordinates the work: it checks authorization, places the request in a queue and routes it to an available compute node where the language model is already loaded in memory.

Inside the hall stand long rows of tall metal cabinets called racks. They hold flat server blades equipped with tightly coupled GPUs or AI accelerators that perform many numerical operations in parallel, right next to fast HBM memory holding billions of model weights. High-speed interconnects link multiple chips and racks together whenever a large model does not fit onto a single card.

• NVIDIA DGX SuperPOD Reference Architecture
• Google TPU v4 Architecture

Two very different workloads run inside an AI data center:

1. Training: Thousands of accelerators work together for weeks or months across massive datasets, gradually adjusting the weights of a new model.
2. Inference (serving): The learned weights sit ready in memory. When your coffee-machine question arrives, the system calculates the most likely continuation token by token and streams the pieces back to your phone.

Nearly every kilowatt-hour of electricity drawn by a compute chip turns into heat. To keep densely packed racks within operating temperature, fans or direct liquid-cooling plates carry that heat away.

PUE (Power Usage Effectiveness) compares total facility electricity with the electricity used by IT equipment: a PUE of 1.10 means that for every 1.00 kWh of compute electricity, an additional 0.10 kWh goes to cooling and power distribution. Whether cooling also evaporates water depends heavily on the cooling design, location and weather.

• Uptime Institute Global Data Center Survey
• ASHRAE TC 9.9 Thermal Guidelines

The International Energy Agency (IEA) and Lawrence Berkeley National Laboratory (LBNL) document rising data-center electricity demand; AI is a major growth driver alongside traditional cloud computing, video streaming and enterprise software.

For individual text prompts, Google published a measurement in May 2025 for a median Gemini text prompt: roughly 0.24 watt-hours of electricity, about 0.03 grams of CO₂e and 0.26 milliliters of water (around five drops). That figure applies specifically to Google's studied fleet, cooling setup and methodology. For other providers, longer conversations, image or video generation, or multi-step reasoning runs, the exact footprint of a single request remains unknown without comparable measurements.

• IEA: Energy and AI Report (2025)
• LBNL: 2024 United States Data Center Energy Usage Report
• Google Cloud: Measuring the environmental impact of AI inference (May 2025)

Next time you see a headline number about AI power or water use, ask three questions:

1. Which workload was counted? Months of training, a short text reply or a multi-step agent run?
2. What is the system boundary? Just the chip, the whole building including cooling, or the local electricity grid mix as well?
3. Is it a measurement or an extrapolation? Does the figure come from actual operating telemetry or a back-of-the-envelope scenario?