Arm CEO Rene Haas just issued a blunt assessment of power needs for AI workloads in data centers globally in a new blog. The rise in AI compute is expected to grow 3x by 2030, more than the total power consumption of India, the world’s largest country.
“Future AI models will continue to become larger and smarter, fueling the need for more compute, which increases demand for power as part of a virtuous cycle,” Haas said. “Finding ways to reduce the power requirements for these large data centers is paramount to achieving the societal breakthroughs and realizing the AI promise. In other words, no electricity, no AI.”
Arm makes chip architectures used widely by chipmakers and naturally puts those designs at the center of the need to reduce power consumption. “It’s no surprise the world’s largest AI hyperscalers have turned to Arm to reduce power,” he added, positioning Arm’s latest Neoverse CPU as best in power and performance for both AI inference and training.
Haas said data centers already consume 460 terawatt-hours of electricity a year, equivalent to all of Germany. In a typical server rack, the compute chip alone consumes more than half the power budget. “Every watt counts,” he said.
He ticked off AWS Arm-based Graviton, Google Cloud Arm-based Axion, Microsoft Azure Arm-based Cobalt and Oracle Cloud Arm-based Ampere Altra Max as examples of Neoverse capabilities. In the Oracle example, Arm-based Ampere helped provide 2.5 times more performance per rack of servers at 2.8 less power versus traditional competition when used for AI inference tasks such as LLM training tokens.
Arm is providing half of the new Nvidia Grace Blackwell platform (GB200), which is made up of the Blackwell GPU and the Arm-based Grace CPU, which reduces energy consumption by 25x with a 30x increase in performance per GPU over existing Nvidia H100 GPUs, which rely on competitor Intel Xeon CPUs.
Haas concluded that Arm deployments could help companies save up to 15% of total data center power, resulting in savings to drive added AI capacity within the same power envelope and “not add to the energy problem.” Such energy savings could run 2 billion more ChatGPT queries or power a quarter of daily web search traffic or light 20 percent of American households, he said.
While the Arm blog is basically an advertisement for Neoverse, it does point to a widely-recognized problem in which data centers trying to launch or expand have faced problems getting power in multiple locations. These include US locations such as Loudon County, Va., parts of Georgia, Texas and Oregon and other locales. Amsterdam has restricted data center growth over power and the problem has more recently confronted Dublin, Ireland.
The International Energy Agency recently said the 2,700 data centers in the US drained more than 4 percent of the country’s total electricity in 2022, with a projection they will consume 6% in 2026.
McKinsey has said US data center demand was nearly 17 gigawatts in 2002 and is projected to reach 25 gigawatts in 2026. It takes a large nuclear plant to generate a gigawatt of power.