A look at AI server growth, to be boosted by chatbots, more

AI has been driving growth in the server market, including the chips inside such servers, as evidenced by strong sales of of GPUs by Nvidia and AMD primarily.  ChatGPT and the fascination with chatbots from a number of major players like Microsoft and OpenAI is accelerating the trend as cloud service providers beef up investments.

Some of the expected growth was recently traced by TrendForce, an analysis firm based in Taipei, Taiwan.  The company said for all of 2022, AI servers equipped with general-purpose GPUs made up just 1% of global server shipments. In 2023, shipments of AI servers are expected to grow 8% year-on-year thanks to chatbots and similar AI applications. Through 2025, the annual growth is AI servers is expected to reach nearly 11%, TrendForce said.

Total global server market shipment volume exceeded 13 million servers in 2022, according to various sources, with analyst firm ResearchandMarkets putting the number at 13.6 million servers, up 5% from 2021. Based on the reporting by both analyst firms, there would have been 136,000 GPU servers shipped in 2022, to increase by some number that could be 147,000 GPU servers shipped in 2023.  It isn’t clear that all or nearly all AI server growth is based on use of GPUs, as Intel and many other small companies are relying on CPU modifications that the companies believe can provision compute for sophisticated AI applications.

IDC has put a dollar value on the overall global server market of $122 billion in 2022, expected to increase 3% in 2023, then hurdle upwards by another 12% in 2024. About 10% of the overall market is for non-x86  servers, while the rest is x86 servers.

It’s been widely reported ChatGPT used at least 10,000 Nvidia GPUs for AI training purposes and TrendForce said those were mainly Nvidia A100 chips, while ChapGPT uses resources and services of Microsoft Azure. The analyst firm tabulated that ChapGPT and other Microsoft apps will result in demand for 25,000 AI servers in 2023. 

Interestingly, Baidu’s ERNIE Bot was originally running A100s, but Baidu switched to Nvidia A800 chips because of US Commerce Department export control restrictions.  TrendForce projected Baidu’s demand for AI servers will total about 2,000 servers for 2023. The A800 is designed specifically for the Chinese market because of the US export restrictions, according to TrendForce. Nvidia now controls about 80% of the server GPU market, while AMD controls about 20%.

Intel GPUs have not gained a significant share in the consumer, non-research focused, market, according to TrendForce analyst Mark Liu. "It may be relatively challenging [for Intel] to end the data center market within two years," Liu added in an email to Fierce Electronics.  In the CPU field, Intel faces "greater challenges" from AMD and the introduction of ARM for servers. 

Four major North American cloud service providers (Google, AWS, Meta and Microsoft) made up 66% of the annual AI server demand in 2022. In China ByteDance had 6% of AI server procurements in 2022, while Tencent, Alibaba and Baidu had 2.3%, 1.5% and 1.5% respectively, TrendForce reported.

The analyst firm also said that AI servers with GPUs require high-bandwidth memory, relying on Samsung, SK Hynix and Micron. Nvidia has adopted HBM3 memory, which means SK Hynix should benefit because it is the only company capable of mass producing HBM3.

An industry-wide  inventory correction for memory  in 2023 will hold back HBM in 2023, but then the annual growth will explode to 40 percent in 2023 to 2025, TrendForce believes.  And, memory suppliers see HBM as providing them high gross margins.

“Cloud companies are going to invest more in AI servers over the years,” TrendForce wrote. “Presently, companies are scaling back IT spending as the global economy is being impacted by high inflation and sluggish growth. However, with applications such as chatbots and search engines driving the demand for AI-based technology transformation, cloud companies will prioritize the related businesses or projects.

RELATED: Generative AI is driving a gold rush for performance tools