Nvidia announced during its GTC21 conference it is working on a new data center chip called Grace, after Rear Admiral Grace Hopper, one of the first computer programmers. The new chip would create a mesh of ARM-based CPUs with Nvidia GPUs and memory.
By connecting the CPU, GPU and memory via higher capacity lanes, the new chip design will be significantly more efficient than current x86 implementations where the connection between CPU and GPU has become the bottleneck.
|
|
x86 with PCIE 4.0 |
Nvidia Grace |
|
GPU |
8000 GB/sec |
8000 GB/sec |
|
CPU |
200 GB/sec |
500 GB/sec |
|
Interface |
16 GB/sec with PCIE 4 |
500 GB/sec with NVLINK |
|
Memory to GPU |
64 GB/sec |
2000 GB/sec |
Source: Nvidia
The 30-fold increase in Memory to GPU performance in Grace will significantly improve high-end AI computation in data centers and represents a massive shift in computational power as AI cores will spend less time waiting for data to arrive before processing it.
The Grace chip is designed with a general ARM cores design available to any licensee and not specially customized to work with Nvidia GPUs. If Nvidia is allowed to buy ARM for $40 billion, the combined company would have many reasons to build specialized CPUs and an integrated architecture that will be considerably more efficient than standard ARM cores, thereby giving the larger Nvidia a more substantial edge in the server market.
The server CPU and chip market is currently undergoing significant change as AMD has been eating into Intel’s market dominance. A few years ago, Intel had 99% of the server chip market, but this has come down to the low 90% range as AMD has been offering superior chips. According to AMD CEO Lisa Su, the company has reached 10% market share in server chips. ARM server chips are still living up to the running joke that “next year is definitely the year of ARM server chips” with around 1% market share.
Several companies have tried to make an impact with ARM chips on the server market. Marvell, Ampere and Huawei have spent significant resources on bringing ARM chips to market that have impressive compute power. In 2019, Huawei even injected $3 billion of funding for companies to build a development platform and ecosphere around it. Subsequently Qualcomm also injected funds to build an ecosphere with limited success, and ultimately left the server market. The results are underwhelming, with the notable exception of custom-build ARM-based chips for captive usage among hyperscalers like Amazon, Microsoft, Facebook and the like.
There are several challenges that Nvidia and ARM would have to overcome should the $40 billion merger proceed:
- The majority of the server chip market uses custom-designed x86 chips with decades of close business relationships, development work by clients on x86, and the lack of a deep and well-known development platform. These types of relationships are hard to impossible to supplant
- Some of the largest data center providers are building their own custom ARM chips based on their proprietary knowledge of workloads. Amazon, Microsoft Google, Facebook, Baidu, Alibaba and Huawei are all reportedly building their own data center ARM chips for their own data centers tailored to their specific and evolving needs. These data center providers need flexibility and value close interaction with core technology providers with their own in-house technology vendors.
- The persistent lack of a native development platform. Even in this cloud-first, emulator-heavy world, developers like to develop on the same silicon as the computer they actually program on. This would require Nvidia to either build a computer that serves as a development platform or tightly integrate with Apple’s ARM-based computers. There is just the slight problem that Apple and ARM are not getting along very well. Nvidia has a long history of building systems (e.g., DGX, EGX, HGX) and could build a development platform. Recently they announced a partnership with Mediatek to enable Nvidia GPUs for Mediatek PC CPUs. Such a development platform could originate from a number of licensees if not Nvidia themselves.
- Change of ARM from being design partner to being a component provider and potential competitor. With ARM being a technology provider that allows others to build chips as the company is a partner of these chipset manufacturers. With Nvidia being in the chipset provider business this relationship is fundamentally altered and due to competitive reasons, ARM will experience walls appearing where none had been before.
But Grace is not the only chip project where Nvidia combines its own technology with that of ARM. Nvidia is in the early phases of deploying Data Processing Units (DPUs) with friendly trial partners for a product called Bluefield-2. The chip combined the SmartNIC from Mellanox, a company that Nvidia acquired in 2020, a CPU by ARM, and an Nvidia GPU. In a year or two, all of these components could be delivered by Nvidia as an integrated solution. A combined Nvidia/ARM has the potential to integrate GPU, CPU, and Smart NIC, to a much greater degree than previously possible, creating a next-generation DPU.
But Nvidia does not need the merger to create a new DPU. ARM’s other licensees will have the exact same ARM technology and will be able to create their own integrated solutions. While Nvidia states that it does not plan to change how ARM operates, anyone that has been part of an acquisition knows that one company does not buy another company in order not to change anything.
In fact, changes are the very reason one company buys another. Even if Nvidia and ARM do not fully integrate, Nvidia, its Mellanox subsidiary, and ARM will have every reason to combine and harmonize road maps that will enhance each other’s strengths and create a new division of responsibilities. Such a reduction of potentially duplicative efforts will on one hand reduce the number of choices available to customers while on the other hand potentially increase the research budget for the solution that Nvidia is favoring. A bigger bang for the buck for Nvidia.
As Nvidia is planning to buy ARM, the 2006 acquisition of ATI by AMD serves as an interesting case study. At the time, AMD reportedly talked to both Nvidia and ATI about an acquisition as a means to better compete with Intel. Ultimately, AMD acquired ATI. The paths of what used to be ATI and Nvidia, close competitors with very similar capabilities at the time of the merger, couldn’t have been more different.
Nvidia successfully leveraged its GPU business into becoming an AI behemoth, and the graphics card division of AMD did not. For a while, AMD struggled to compete with Intel while AMD’s graphics card division kept the company afloat. Both AMD and Nvidia make excellent gaming graphics cards with performance roughly on par, but the independent Nvidia does so much more, as is evidenced by its market cap. AMD’s market cap is roughly $95 billion including the value of the graphics processor division it bought instead of Nvidia, while Nvidia is worth $355 billion by itself. Meanwhile, Intel, the boogieman that drove AMD into buying ATI, has incorporated basic graphic functions into the CPUs, but even 15 years later is not competing with AMD and Nvidia in the discrete graphics card market. Intel’s market cap is around $230 billion. It is clear that Nvidia made the right choice of remaining independent.
Like so many things when it comes to anti-trust, the question and answer lies in how you define a market. If you look at the overall CPU market, it is huge and diversified. If you look at it more closely you see three segments with an overlay of AI: Mobile is dominated by ARM, the data center market is dominated by Intel, and the PC market is split 55/40/5 between Intel, AMD and ARM. Nobody really plays in all segments, which creates various perspectives and incentives:
- The server space is heavily dominated by Intel with a custom design ARM segment from hyper scalers. Parties that want to bring competition to this segment would probably support Nvidia in order to get more investment and integration in this space.
- In the PC space, ARM is the newcomer that is gaining share through Apple. AMD is also eating into Intel shares as Intel’s CPUs have not been as successful as they have been in the past. Companies like Apple who want to create their own silicon and not get coaxed into buying an integrated solution from a combined ARM and Nvidia would be very skeptical of such a combination
- In the mobile space, ARM is the de facto standard and ARM has historically been very easy to work with. Any change is viewed with great skepticism and a likely negative outcome.
- In the AI space, the only company that poses a viable threat to Nvidia is ARM. Nobody else comes close.
The reaction of the different companies who have publicly commented can be directly linked to where they play in the ecosphere. The new Nvdia with ARM would be the only player in all four segments.
Fundamentally, decisions and acquisitions have consequences. They change the trajectory of companies and they change the future. Regulators have to decide what kind of future is more important.
A future with ARM as an independent company focusing more on a RISC CPU, device-level AI and graphics would have at its core the mission that has created a huge ecosphere that enables dozens of companies to build various CPUs, AI chips, and graphics processors resulting in more than 180 billion ARM processors being produced.
Alternatively, there could be a different future where ARM works foremost with its new owner Nvidia and builds a more data center-centric world, where the interests of the ecosphere at large will necessarily take the backseat.
Roger Entner is the founder and analyst at Recon Analytics. He received an honorary doctor of science degree from Heriot-Watt University. Recon Analytics specializes in fact-based research and the analysis of disparate data sources to provide unprecedented insights into the world of technology. Follow Roger on Twitter @rogerentner.