Intel’s third Innovation event allowed the chip giant to introduce work in multiple areas for developers, but the AI PC and the Core Ultra (Meteor Lake) chip with its Neural Processing Unit stood out for me.
It was ironic for Intel to talk about a fundamental new concept for PCs that can run nimble AI (with, say, 15 billion parameters per model instead of upwards of 1 trillion parameters per generative AI model in the cloud), even amid a 14% decline expected in the PC market in 2023—the largest decline in 40 years, as measured by IDC. Some developers may be waiting for the Acer Swift laptop with Meteor Lake, but it really isn’t possible to understand how much developer demand there will be.
RELATED: PCs will decline in ’23 by the most in 40 years, IDC says
Nobody in the developer community, really, has said what apps, specifically, will be the most popular to be created with nimble AI on an AI PC, but it is the ultimate question. (A keynote onstage demo of Rewind AI’s capabilities was, however, a good clue.)
RELATED: What’s an Intel AI PC good for?
Does Intel have some kind of developer intuition about the need for this, or even a survey? They answer that question sometimes with a contrapositive, which is that big AI chips (GPUs like Nvidia’s H100) can cost a fortune (maybe as much as $10k apiece) and are still selling like hotcakes if a supply can be found. They are for cloud systems, not the PC.
Meteor Lake ships for PCs Dec. 14, and Intel says it will have a version of it, ruggedized, for widespread distribution to edge devices in 2024. That could be billions of chips that would help retailers and medical devices function with a lightweight AI capability that is dependent on retrieval-based data, not from the AI model’s memory. One idea is a Meteor Lake implementation in a factory where machines are monitored for early indications of breakdown.
RELATED: Intel and nimble AI: smaller models can run on an AI PC
RELATED: Intel to expand AI Meteor Lake chip to edge, beyond the AI PC
Several analysts have decreed the AI PC as vastly important, and that comes from analysts who are not apologists for Intel and have dinged the company repeatedly for falling behind in the past on its chip cycle. At Innovation, CEO Pat Gelsinger said Intel was on course for “five nodes in four years” so many times that he should have worn it like a sandwich board as he ran to multiple meetings.
“Generative AI being embedded in PCs is a watershed moment,” said IDC analyst Linn Huang via email. “Personally, I think AI will be hugely transformative to the PC user experience and as impactful to the PC market. It might not drastically increase the long-run total addressable market, but it should provide a significant uplilft to pricing and could reshape competitive balance in this industry over the next decade.”
Remember, IDC Is the outlet that noted the biggest decline in PC’s shipped (14%) in a single year in the past 40 years. IDC says the industry segment will grow by 4% in 2023, but still behind 2019 pre-Covid levels. Growth in 2024 will have more to do with the next major refresh cycle and Windows 11 migration than AI.
Huang said it would probably take two years for Microsoft to fold in more AI features into Windows and as silicon players work on ramping up of supply for AI on PCs. “In the near term, we see exciting product launches and huge marketing campaigns for AI on PCs, but our volume expectations are low given supply chain limitations,” Huang added.
Also, Huang noted, by the time AI PCs are shipping in larger volumes, Intel may have Lunar Lake out. That’s the next generation Core chip after Meteor Lake (careful where you swim) due in 2024 or early 2025, which is expected to focus on low power consumption and performance per watt.
Analyst Leonard Lee at neXt Curve made the argument that the focus on Intel’s AI PC overlooks Apple’s AI Mac, which has really been around since the introduction of Apple silicon. “In order to keep the Windows PC ecosystem competitive, Intel and Microsoft needed to make the AI push,” he said in an email. “Certainly, the frequently stated hope by Intel and Microsoft leadership is that the AI PC will drive a strong upgrade cycle among consumers and enterprises and revitalize their collective OEM ecosystem. It is pretty obvious that the two companies are looking to ride the generative AI hype wave to turn the tide in their client businesses which have seen negative revenue trends over the last few quarters.”
Apple Macs have made strong strides over the past three years, even while sales have been down lately year on year. From 2019 to 2022, Mac saw a 60% increase in market share while Intel PC share grew 6%. Still, Macs made up only about 11% of all global PC shipments in early 2023, and 17% in the U.S. Macs tend to float from 8% to 13% of the global PC market shipments depending on proximity to a launch of a new machine in any given quarter.
Huang said everybody will want to make an AI PC chip if they haven’t already. Apple is building a neural core into its silicon, as Lee noted, although Apple has not discussed its specific AI strategy openly thus far. Qualcomm’s next Snapdragon for Windows is said to have very advanced AI performance. AMD has been talking about AI on Ryzen since CEO Lisa Su mentioned it at CES in January. “It’s all moving very fast,” Huang said.
For generative AI, cloud-based services are moving much faster than client-based ones in development. It almost seems like a mouse compared to an elephant at this point. However, Huang said one argument for AI on the client computer is the need to alleviate potential resource crunches.
“If growth of things like ChatGPT continues to outpace the growth of GPU core shipments, there will be a need to put as much AI in the edge and on the client to keep pace,” Huang said. “AI on PC is also resonating with business buyers because of the issue of data sovereignty.” No business wants to allow company data to be fed into a larger model, which could be potentially accessed by others for some benefit.
“In the future, the best way to keep data internal might be to keep AI off the cloud,” Huang said.
Jack Gold, an analyst at J. Gold Associates, also sees plenty of positives in an AI PC.
"If you put AI into the PC, apps and workloads will come," Gold said, recalling when the industry first saw integrated graphics going into processors, with the integrated graphics much less capable than external GPU graphics processing. "But new usage models came and it became an expected capability. The same will be true of AI."
Apps will include enhanced security detection and response, guided use of OS and apps, enhanced search, better graphics/audio generation, ease of looking up how to do something on the machine, concierge services watching what you are doing and making recommendations, Gold said.
"All that could take place at the endpoint rather than ship everything to the cloud. As AI use picks up, the load on the cloud will increase dramatically, so having more capability locally will be a major issue. That’s even more true at the edge, where Meteor Lake and successors will also be deployed. The PC is an edge device in many situations."
"Yes, the PC market is shrinking, at least temporarily, but AI and especially AI inference that runs models trained on a higher end system will become commonplace in the next one to two years," Gold predicted.
"That change will require AI acceleration units like Intel's NPUs and all the vendors will have them. And the compelling and advanced functionality apps will be coming, as well as AI-enhanced OS capability from Microsoft, Apple, and others. Even iOS and Android will have AI functions accelerated through NPU in mobile chips."
By the way, AI PCs will probably cost more than traditional PCs, said IDC analyst Jitesh Ubrani. But we expected as much, right?
Having heard a lot about the AI PC in recent days, I’m eager to hear what actual developers think. Could it solve problems for your operation, small or large? Please email me your thoughts at [email protected]