Chiplets are small, but they’re enabling big things in the computing world.
Their adoption has seen a steep uptick in recent years, thanks in large part to the exponential growth of artificial intelligence (AI) that’s putting increasing pressure on servers, high-performance computing (HPC), and data centers.
Rather than consolidate every part into a single chip using a monolithic approach, chiplets are segmented with specific segments manufactured as separate chips, which are mounted into a single, interconnected package. A key benefit is that the different parts within a chiplet can leverage the latest fabrication methods and be shrunk in size so more components can be squeezed in.
Chiplets allow for mixing different processes, thereby providing the best of both worlds, Jim Handy, principal analyst with Objective Analysis, told Fierce Electronics,. High Bandwidth Memory (HBM) could be viewed as a chiplet approach, he said, while Nvidia’s Blackwell processor has two GPUs intricately tied together.
“In the long run, it presents an opportunity to plug in emerging memories,” Handy said. A system-on-a-chip (SoC) could use MRAM, ReRAM or FRAM, which he said is beneficial given the scaling limits of NOR flash and SRAM. “SRAM is not shrinking in proportion to the size of the logic transistors.”
Handy said the recent introduction of standards by the Universal Chiplet Interconnect Express (UCIe) Consortium demonstrates that chiplets are headed in the same direction as regular chips as having a standard interface for multiple customers means a chiplet can be supplied for cheaper than a custom part. “If you use UCIe then it gives you the opportunity to have multiple sources for the chiplets that you're buying.”
UCIe was conceived in March 2022, with the first iteration of the standard supporting 2D and 2.5D – a planar interconnect which meant chiplets were side by side. The recently released UCIe 2.0 supports 3D stacking of chiplets vertically by addressing the required connectivity while nearly eliminating the distance between chips.
Ecosystem gains momentum
In parallel with the introduction of a chiplet standard, there are companies enabling the ecosystem such as Baya Systems. It has an algorithm-driven system architecture platform combined with its scalable IP and cache fabric that pulls together all the steps of building out chiplet architectures. Eliyan Corporation, meanwhile, is offering its “bunch of wires” (BoW) chiplet system as a more efficient approach to packaging.
Robert Patti, president and CEO of NHanced Semiconductors, said there’s no question that everyone sees chiplets at the way forward, but he sees the UCIe standard as addressing a niche market, albeit a large one. “The type of devices that work with UCIe are going to be smaller geometry and leading edge. It's easy to see how UCIe fits well into dense compute. We're going to see a UCIe set of HBM-like memories crawl out of the woodwork in the near future.”
However, it won’t work well in other areas, such as legacy nodes, Patti said, even though it’s a big chunk of the market. “It is not the be all end all solution.”
He believes the BoW approach has “got legs” and will help address the legacy nodes and applications outside of the compute space as it’s not as complex as UCIe. “We're going to start to see ad hoc standards that will work their way through the marketplace and pop out at the other end.” These will be best practices, he said, not de facto standards.
NHanced is focused on building special purpose chiplets for specific customers and design components for tasks the customer wishes to accomplish. “It's not meant for a broad set of applications that seems to be where most of the chiplets are today.” Patti said the company’s customers are turning to chiplets because they’re looking at a future where they can do more with greater flexibility and time to market at a lower cost.
Patti said the ultimate leverage in chiplets is by going to more specific markets. “What we now need to do is take transistors out of devices to reduce cost, which means that we're targeting applications.”
Chiplet adoption spans many verticals
Research firm IDTechEx is predicting the chiplet market will reach $411 billion by 2035, driven by high-performance computing demands across sectors such as data centers and AI. Examples of high-performance computing implementations include companies like AMD and Intel, which have adopted chiplet designs in products such as AMD's EPYC processors and Intel's Ponte Vecchio GPUs.
Aside from servers, IDTechEx sees chiplets playing a crucial role in telecommunications including 5G and internet of things (IoT) to enable efficient network solutions, PCs, mobile phones, and integrating diverse functionalities for automotive applications.
Eddie Ramirez, VP of Arm’s infrastructure business, told Fierce Electronics in an interview that he sees a lot of adoption in server CPUs. “The majority of them are leveraging chiplets in some form,” he said. “The size of the silicon is just so large that the economics means that you want to break it into different parts.”
The company recently announced it was partnering with Samsung Foundry, ADTechnology and Rebellions to bring to market an AI CPU chiplet platform that targets cloud, HPC, and AI/ML training and inference workloads.
Ramirez said one of the challenges with the initial wave of chiplet designs was they were being built with proprietary interconnects. “What we have seen is that the market has moved towards UCIe as opposed to these proprietary interfaces,” he said. “There are multiple partners that are delivering UCIe IP and they're available on all the major foundry nodes now. The industry seems to now be accepting that as the physical interconnect.”
What Arm is doing is providing a protocol layer on top of the physical layer so that everything lines up with UCIe, Ramirez said, which is helping to accelerate chiplet designs.
Chuck Sobey, chief scientist at ChannelScience and general chair of the Chiplet Summit that is now going its third year next January, told Fierce Electronics that going forward interoperability of chiplets is critical, and the UCIe standard as well as the Open Computer Marketplace are important steps. Like Ramirez, he sees server chiplets getting the most adoption, but the automotive sector is seriously eyeing chiplet technology as well.
Sobey said the automotive vertical will require a chiplet ecosystem that supports long lead times and its stringent requirements.
In the meantime, the data center is changing with explosive growth of generative AI, he said, which has had serious ramifications on power, and chiplets are an important piece of the puzzle to increase density. “The generative AI data center is going to generate that data for you instead of looking for it and serving it up to you,” Sobey said. “It takes more energy to move the data than it does to compute the data these days.”