Updated from Friday...
For much of 2024, Wall Street observers have been cautiously monitoring the AI market for signs of a reckoning with the explosive exuberance that has come to characterize the sector, and has driven the valuations and prospects of almost any company associated with AI–but especially semiconductor firms–through the roof. That reckoning might have arrived this week, as companies ranging from Meta and Microsoft to Nvidia and AMD and Super Micro Computer saw their stock values plummet Thursday.
Is it related to the generally positive, but somewhat cautious guidance issued by companies like AMD and Microsoft during their earnings calls this week? Does it have something to do with the AI market’s migration from a training phase to an inference phase? Has there been a lot back-end spending on compute resources to train AI, and just not enough evidence yet of front-end results? Is there still just too much uncertainty about AI and what it can or can’t do?
All of these things are true to some extent. AMD posted strong earnings in its Q3 and now expects to reap about $5 billion in revenue this year from AI-related devices like its MI300X Instinct GPU. Meta and Microsoft also reported great quarters driven largely by AI advances (Even Intel touted AI-related growth.) But investors and analysts questioned the current-quarter guidance provided by both AMD and Microsoft, with both viewed as having issued somewhat soft forecasts.
Meanwhile, Meta provided what was generally viewed as strong guidance, but also talked a lot about how AI is driving increasing capex for the company. Meta CEO Mark Zuckerberg told analysts that the return on investment will be worth it, but that it is increasing AI-related spending because “our AI investments continue to require serious infrastructure.”
Also, there continues to be a lot of uncertainty around AI. Giants like Microsoft, Meta, and Amazon are using AI both internally and externally through chatbots and other projects that require large language model training, and model training requires a lot of computing power. That is driving investment in the latest AI processors and accelerators, particularly GPUs, from Nvidia, AMD, and others. Wall Street understands this and has rewarded Nvidia in particular.
But what happens next? The majority of AI resources so far have been put to work on AI training, but AI inference is expected to represent a bigger part of the AI play in the near future for companies that are pretty far along in their development of AI applications.
Jack Gold, president and principal analyst at J. Gold Associates, has noted how this could change the mix of computing resources required. He reiterated to Fierce Electronics this week, “Inference changes the mix of chips being deployed. Inference can actually run on smaller, or at least less complex GPUs that often can be integrated in with the CPU. It's why all the major chip vendors have NPUs attached to their latest processors (e.g., Intel, AMD, Qualcomm, and even ARM IP).”
Gold added, “We’ve been saying for a while now that in the next 2-3 years, we’ll see 85%+ of AI workloads move from training to inference, including running inference at the edge. That fundamentally changes the mix of processors, and moves away from the massive GPUs like Blackwell, and towards more integrated AI accelerators attached to CPUs and more general purpose processors that can run a variety of different workloads.”
The need–and the market–for chips used for AI model training will not suddenly dry up as this transition plays out, but it has created some uncertainty. Industry watchers are waiting to see when and how hard the other shoe drops, and who could be negatively impacted as a result.
Gold said, “Nvidia doesn’t have a great story here as its CPUs aren’t in the same league as Intel/AMD. So Nvidia will likely see a slide in the coming years. Training won’t be dead, so they’ll still sell big chips. But a lot of AI workloads will move off the massive GPUs.”
That could lower the volume of excitement around Nvidia, its partners, like Super Micro Computer, as well as the throng of companies that have emerged with alternative GPU-focused business models, such as GPU rental plans. While AMD may be well positioned, the company also has been spending a lot of time recently celebrating market traction for its latest call, CEO Lisa Su expressed confidence in the ongoing market opportunity for its GPUs and how well they can handle inference, while also making it clear that processors like AMD’s EPYC CPU family are ready to tackle inference as well.
Along with this transition from training to inference, other analysts have pointed out that there has not been much evidence yet that early enterprise AI projects and trials are progressing to larger commercial deployments. There is now hype around AI chatbots and so-called “agentic AI,” but it is still early. Leonard Lee, executive analyst and founder of neXt Curve, recently told Fierce Electronics that enterprise generative AI is “nascent,” and that enterprises are still learning.
The market is changing and maturing, and industry watchers and investors continue to evolve their understanding of it. AI hype may not be over, but it took a little bit of a rest last week.