Inside the relentless race for AI capacity

These spikes threaten cascading power outages, affecting homes and businesses that feed off the same grid network. Last summer, utility providers in Virginia had to grapple with a sudden surge in power after a cluster of facilities switched to back-up generators as a safety precaution, leading to an excess supply that risked grid infrastructure.

With abundant power the priority, operators have also ended up in areas with significant water constraints. Hyperscale and colocation sites in the US consumed 55bn litres of water in 2023, according to researchers at the LBNL. Indirect consumption tied to energy use is markedly higher at 800bn litres a year, the equivalent annual water usage of almost 2mn US homes.

In 2023, Microsoft said that 42 per cent of its water came from “areas with water stress”, while Meta said roughly 16 per cent of its water usage was derived from similar areas during the same time period. Google said last year almost 30 per cent of its water came from watersheds with a medium or high risk of “water depletion or scarcity”. Amazon does not disclose its figure.

Data centres in drought-prone states such as Arizona and Texas have led to concern among locals, while residents in Georgia have complained that Meta’s development in the state has damaged water wells, pushed up the cost of municipal water and led to shortages that could see water rationed.

Some believe that this endless race for ever-greater computing power is misplaced.

Sasha Luccioni, AI and climate lead at open-source start-up Hugging Face, said alternative techniques to train AI models, such as distillation or the use of smaller models, were gaining popularity and could allow developers to build powerful models at a fraction of the cost.

“It’s almost like a mass hallucination where everyone is on the same wavelength that we need more data centers without actually questioning why,” she said.

Increased chip density has another unwanted effect: heat. About two-fifths of the energy used by an AI data centre stems from cooling chips and equipment, according to consultants McKinsey.

Early data centres running cloud workloads deployed industrial-grade air conditioning units similar to those used in offices to cool servers. But as chips started to draw more power, it has become harder to keep them within their safe operating range between 30 and 40C, with data centres requiring more advanced cooling methods to avoid malfunctions. The challenge has led to significant investment in cutting-edge innovations.

Operators have turned to installing pipes filled with cold water in the server room to transfer heat away from equipment. This water is then directed to large cooling towers that use evaporation to reduce the facility’s temperature to a safe range. But the approach leads to water loss, with a single tower churning through about 19,000 litres per minute.

Microsoft and others have adopted a closed-loop system that depends on chillers — in effect, a refrigerator —to cool the water. This process is less wasteful and more efficient than evaporative options.