Power consumption with AI chips is one of the top concerns on the minds of data center gurus and server manufacturers like Dell.
Dell COO Jeff Clarke made that point again during his company’s earnings call last week, when he revealed that Nvidia’s latest AI accelerator will consume 1,000 watts, up by 42% from its predecessor.
Even so, he said the upcoming Nvidia chip he called the B200 won’t require liquid cooling in data center servers. Nvidia has not announced a chip called the B200, but Clarke could be referring to the GB200 chip on Nvidia’s roadmap, some observers speculated, which is expected to combine Nvidia’s Grace CPU and its B100 GPU, the same configuration as the GH200. GB200’s thermal design power could be as much as 1,300 watts, an improvement of 30 percent over its predecessor.
Nvidia may reveal more at its GTC event in two weeks.
Nvidia announced the H200 at the end of 2023, an update of the H100, with the ability to double LLM performance because of its HBM3e memory.
On the Dell earnings call, Clarke said, “We’re excited about what’s happening with the H200 and its performance improvement.” Then he said, “We’re excited about what happens at the B100 and B200 and think that’s where there’s actually another opportunity to distinguish engineering confidence. Our characterization in the thermal side, you really don’t need direct liquid cooling to get to the energy density of 1,000 watts per GPU.
“That happens next year with the B200,” he said. He added that Dell has shown expertise in making liquid cooling perform at scale, interconnect work, telemetry and power management. “It really allows us to be prepared to bring that to the marketplace at scale to take advantage of this incredible computational capacity or intensity or capability that will exist in the marketplace.”