Nvidia CEO Jensen Huang introduced a dizzying array of products at the GTC24 event on Monday, including a Blackwell platform to run generative AI on LLMs with one trillion parameters and 25 times less cost and energy than its predecessor.
Blackwell will thrill the hardcore cloud and data center managers at the Silicon Valley event, but Huang also unveiled a humanoid robot project called GROOT, and described how Apple’s Vision Pro headset for spatial computing can be used with Nvidia Omniverse digital twins to design cars and other products.
Huang continued to show his company is on an AI tear, generative or otherwise, burning up financial markets with the company’s top market cap, bouncing around China export rules and blasting through previous silicon limits.
Another important new product from Nvidia called NIM is designed to provide a runtime environment for AI microservices that enterprises can use for inference work on two dozen Nvidia AI models and its partner ecosystem. Huang said enterprises are “sitting on a goldmine of data” that can be converted to generative AI and become building blocks to their becoming AI companies.
First day GTC2024 highlights
Project GROOT is a model for humanoid robots, which Nvidia is producing alongside Jetson Thor, an SoC and upgrades to Nvidia Isaac. Nvidia said GROOT robots will understand natural language and emulate human actions to learn dexterity. Jetson Thor runs a GPU based on Blackwell that can provide 800 teraflops of AI performance in 8-bit data processing.
Nvidia said it is building a comprehensive AI platform for leading humanoid companies: 1X Technologies, Agility Robotics, Appronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics and XPENG Robotics, among others.
Apple Vision Pro has been on the market a while and now Nvidia software lets developers bring OpenUSD-based Omniverse enterprise digital twins to the headset. In a demo, a digital twin of a car was streamed to the headset in high resolution. Car configuration developed by Katana on Omniverse allowed the designer to enter the virtual vehicle and make design changes.
Nvidia told reporters in a briefing that the application relies on RTX cloud rendering which are streamed to Apple Vision Pro. A hybrid rendering approach combines local and remote rendering on the device, using Nvidia GDN, a cloud-to-edge streaming approach where the heavy compute is done in the cloud.
Nvidia's new Blackwell GPU, named after mathematician David Blackwell, succeeds Hopper, announced two years ago. It has 208 billion transistors and is being manufactured in a custom 4NP process by TSMC .
Meant to enable AI training and LLM inference for models up to 10 trillion parameters, it relies on Fifth Gen. NVLink and a Second Gen. Transformer Engine. Amazon, Google, Meta, Microsoft and OpenAI are planning to adopt Blackwell.
Nvidia also announced its next supercomputer, a DGX SuperPOD, to be powered by Nvidia GB200 Grace Blackwell Superchips for processing trillion-parameter models. It totals 11.5 exaflops of compute power.
The SuperPOD is made up of 36 Nvidia GB200 superchips, which are made up of 36 Grace CPUs and 72 Blackwell GPUs that are connected via Fifth Gen. NVLink. This combination affords 192 GB of HBM3E memory. With the left and right brains operating as one, a programmer can run every CUDA instance, said Ian Buck, vice president of Nvidia high performance computing, in a briefing with reporters.
Performance of the GB200 Superchips is 30X the performance of H100 Tensor Core GPUs. With Quantum Infiniband, tens of thousands of GB200 Superchips can be connected. The DGX SuperPOD and GB200 and B200 will be available later in 2024.
Blackwell has 25 times better energy efficiency of the prior generation, Buck said.
“There’s definitely an industry effort [in AI chips] to focus on performance per watt and lower that thermal envelope and power requirements,” said Olivier Blanchard, research director at Futurum Group. “This is a trend Nvidia is jumping on as well.”
Analysts at SemiAnalysis described iternations of the Blackwell chip, including the B100, B200 and GB200, with the B100 (code-named Umbriel) air-cooled with 700 watts, and the B200 at a power limit of 1,000 watts, which the analysts claimed can still be air-cooled. The GB200 is the product most anticipated.
In addition to hardware, Nvidia has long made a pitch for its capabilities in software to run on its chips. At GTC2024, Nvidia introduced NIM microservices, a runtime for optimized inference. This approach is designed to allow enterprises to create and deploy custom apps on their own platforms, while retaining full ownership of their IP.
NIM microservices will be free with Nvidia AI Enterprise 5.0 running on Nvidia certified systems (from the likes of Cisco, Dell, HP, HPE, Lenovo and Supermicro) and with leading cloud markets. Enterprise 5.0 will be priced as before at $4,500 per GPU per year or $1 per GPU per hour.
“Inference will proliferate in the cloud and even workstations going forward,” said Manuvir Das, vice president of enterprise computing at Nvidia. “The Nvidia GPU is the best place to run these models, we believe, and NIM is the best software package and best runtime to focus on enterprise apps.”
CES: Nvidia spotlights AI impact on gaming, automobile, robotics