Nvidia CEO Jensen Huang sought to assure customers there will be enough AI capacity to meet their AI demands through 2023.
“We are going to race as fast as we can to meet the delivery demands of our customers, but we have ample supply for the year,” he told reporters in a call during the company’s GTC Spring 2023 event on Tuesday.
The comment came in response to concerns raised by analysts that GPUs from Nvidia face a one-year lead time from the time an order is made to the time the order is fulfilled.
“The demand for Nvidia AI computing has accelerated tremendously since ChatGPT. Generative AI has added to that,” he told reporters in the Asia-Pacific region on Tuesday, according to a transcript provided to Fierce by an Nvidia spokeswoman.
Huang added: “Generative AI now has languages, images, videos, proteins and chemicals. The number of industries that can be helped with AI has grown tremendously. So the combination of all of these factors has accelerated the demand of Nvidia AI, for both training as well as inference.”
During a separate call with reporters in various countries on Wednesday, Huang also said Nvidia’s Grace CPU and and its Grace Hopper CPU-GPU superchip are in production. “Silicon is flying through the fab now and systems are being made,” he said. “Both are in production and being sampled and software is being ported on it.”
He challenged a reporter who said both products were expected to be shipping by about now, adding somewhat whimsically, “go easy on the engineers; there were challenges that led to that [delay].”
Earlier at GTC, Nvidia announced two new chips for inferencing work, the L4 for AI video GPU and the H100 NVL with dual-GPU NVLink for large language models expected in the second half 2023. The H100 GPU that forms the basis of the H100 NVL is now shipping. Huang also introduced the latest of its DGX supercomputer with eight H100 GPUs linked together to work as one giant GPU.
Separately, Reuters reported that Nvidia had modified the H100 to comply with US export rules so the chipmaker could sell the altered H100 as the H800 to China.
In 2022, the US promulgated export rules that prevented Nvidia from selling its A100 and H100 GPUs to Chinese clients. The rules said GPU exports must have chip-to-chip data transfer rates below 600 GBps. On the A100, Nvidia trimmed the GPU’s 600 GBps down to 400 GBps and rebranded it as the A800 for the Chinese market. A similar approach is being used with the H100.
RELATED: Nvidia beefs up AI work at GTC with new GPUs for inference
While the question of lead times for access to GPUs is critically important to many customers, the impact can vary. During the height of the pandemic, some carmakers saw lead times for power chips reach well beyond a year, which hindered shipments of trucks and other vehicles.
Dylan Patel, an analyst at SemiAnalysis recently told Fierce that there is currently a “huge supply shortage of Nvidia GPUs and networking equipment from Broadcom and Nvidia due to a massive spike in demand. He said both companies are ramping up quickly, with a one-year lead time across products. He said specific customers have told him there is a shortage of H100 HGX, forcing them to wait a while to get all the ones they need.
The biggest bottlenecks are across high bandwidth memory, networking and CoWoS, he said. CoWoS is an integrated circuit for high performance computing produced by TSMC.
At the outset of the GTC event, Ian Buck, vice president of Nvidia's acclerated computing unit, assured Fierce there was no problem obtaining GPUs from TSMC. Like many companies, Nvidia does not fabricate its chips and instead designs them and contracts with TSMC and others for chip manufacturing.
Jack Gold, an analyst at J. Gold Associates, said Huang may have been saying the AI capacity customers need could be coming from access to cloud resources and not only GPUs. Nvidia did not elaborate on Huang's statement, however.