Where your AI answer is computed

Our example is a chat question: “Explain why importing an empty file fails.” The application sends that question and the necessary conversation context to a service. It looks like a single action on screen. Several systems work behind the interface.

This journey is a simplified example of a cloud-based AI application, not a map of a particular provider. Some models run locally, and systems can distribute the work differently. A cloud is not disembodied computation: the participating computers exist somewhere.

1. Application: Receives your question and permitted context. Assembles a request and sends it to the service.
2. Service: Receives the request and access authorization. Checks and routes work; model input goes forward.
3. Compute system: Receives model input and loaded parameters. Calculates a continuation; pieces of answer text come out.
4. Application: Receives and displays those pieces. The result is an answer you can verify.

First, the data has to arrive. Network equipment transports requests between devices and services. Inside a data centre, it connects servers, storage and compute nodes. A server is a computer doing work for other devices or programs.

In our example, a front-end service checks whether the request is allowed and where it should go next. When many requests arrive together, the system must coordinate free capacity, queues and the requested model. That is work before the first visible answer appears.

Think of a restaurant kitchen: an order must be accepted and assigned to a work station. A bigger kitchen is of little help if orders sit in the wrong queue. This illustrates routing and waiting time; it does not describe any particular AI provider.

Training gives the model many numerical parameters. For the current request, the compute system uses them to calculate an answer. It first splits the question into tokens: units that can represent words, parts of words or other character sequences, depending on the model. A token is not necessarily a whole word.

A common autoregressive language model has two important phases: prefill processes existing input; decode produces the continuation step by step. GPUs accelerate many numerical operations in parallel. Compute alone does not determine speed; moving required data through memory matters too. Technical process.

Our import question can therefore take different amounts of work. The input may contain one short error message or a large code project. The output may be a short explanation or a long proposed solution. “One request” is not a fixed quantity of computation.

Generating the answer requires intermediate information. A KV cache retains particular calculations associated with earlier tokens, avoiding complete recomputation at each new step. This cache is not an evidence archive or permanent personal memory: it supports the current computation. How this cache works.

Longer context or more simultaneous conversations can change memory requirements. Two questions sit together: how much can the chip compute, and how quickly can the right data reach it?

Multiple compute nodes may also need to exchange information. A large number for one chip does not tell the entire story of a service. Compute, memory, connections and scheduling software must work together.

These devices need electrical energy. They generate heat during operation, which must be removed. A data centre therefore contains more than compute chips: power supplies, networking, storage and temperature-control systems belong to it too. The IEA explicitly separates these components in its energy analysis. Data-centre components.

Power describes the instantaneous rate of energy use, for example in kilowatts. Energy describes use over time, for example in kilowatt-hours. A higher power connection is not a measured annual consumption figure.

Water and CO₂e are separate quantities. Climate impact depends partly on electricity generation and the system boundaries considered. Water use depends on the cooling method and local circumstances. The visible length of a chat answer cannot establish these quantities.

Power usage effectiveness, or PUE, compares total facility energy with the energy used by IT equipment: PUE = total energy ÷ IT energy, for the same period and boundary. It helps describe additional infrastructure energy alongside the IT load. Definition and measurement scope.

A better PUE does not automatically mean less IT energy per model answer. The ratio also does not contain the electricity mix, water use or answer quality. Unknown IT energy or incompatible measurement boundaries cannot produce a meaningful result.

For energy per chat question, we additionally need a measured or defensible modeled allocation of shared resources: model, input, output, hardware, utilization, cache use and time. Missing data means the value is unknown. It is neither zero nor a reason to invent one apparently exact universal figure.

Infrastructure does not stop at buying an accelerator. New servers need suitable power connections, cooling and networking. Shared systems must coordinate many assignments reliably. An announced capacity is not an operating facility.

Two announcements from 12 October 2026 illustrate the distinction. AMD introduced Helion as an open rack blueprint, with partner configurations planned for November. NVIDIA and Cisco announced USFP as a planned optical module format. Both concern infrastructure designs, not a completed data-centre fleet measured by us. AMD announcement, NVIDIA/Cisco announcement.

A more capable model, more simultaneous users and longer agent assignments place different demands on infrastructure. An agent can make many model calls and tool steps. A completed unit of work is then different from one chat response.

Our calculated text pieces return to the application. You read an explanation of the import failure. Its correctness must be checked against the code and appropriate tests. The physical computation path does not make the content automatically true.

You now know the journey behind Enter: accept the request → distribute the work → process input → generate output → return data. Electricity and heat accompany the devices; networks and memory connect the steps.

When you see the next AI-infrastructure number, ask: is it a specification, planned capacity or measurement? Which period and equipment does it cover? Is it about a whole facility, one system, or really an individual assignment?