As AI explodes, are there enough GPUs to go around?
That is a question first raised last year, as Nvidia in particular reported explosive demand for its GPUs that reportedly out-paced supply. Though Meta and other large GPU customers have more recently said that the supply crunch had eased, it highlighted a challenge: As companies large and small continue to need more and more chips for AI developments, it has been getting harder for some of them–particularly the small customers–to get what they need without blowing up their bank accounts paying escalating prices. It is an issue made more difficult by the fact that deep-pocketed companies such as Andreesen Horowitz reportedly have been stockpiling GPUs for use by the partners or portfolio firms, according to a report by The Information.
This problem has given rise to some interesting new models for making AI chip capacity. For example, Akash Network allows users to rent GPU capacity from its network on an hourly basis. Coming into the mix with a similar idea is NetMind.AI, which has a service called NetMind Power that harnesses the capacity and computing power of idle GPUs from around the world into decentralized clusters, that users can access from its global distributed network at far lower cost than buying their own GPUs.
NetMind CEO Kai Zou recently spoke with Fierce Electronics to explain what NetMind.AI's approach is and how it can make a difference in the market.
Fierce Electronics: How challenging is it for smaller companies to obtain GPU computing resources, particularly as giant companies have eaten up supply?
Kai Zou: The increasing complexity of AI models has sent the demand for GPUs skyrocketing. The Big Tech arms race of amassing as many chips as possible has highlighted the fact that GPUs are a limited resource. This scarcity has led to a surge in costs, making cloud access to GPUs prohibitively expensive for small companies that do not have access to the infrastructure owned by Big Tech. Combine this with Nvidia’s Blackwell GPU, priced between $30,000 and 40,000 [Editor’s Note: These figures were mentioned by Nvidia CEO Jensen Huang in early 2024.] and you can immediately see that high cloud service fees are largely unaffordable for smaller firms.
The growing costs and limited availability of cloud GPUs create bottlenecks in innovation and development, making it difficult for smaller players to compete on an equal footing. Our decentralized approach at NetMind Power aims to address this imbalance by democratizing access to AI resources, thus enabling broader participation in AI development.
FE: What is meant by a "decentralized" GPU approach?
KZ: A "decentralized" GPU cluster refers to a network where GPU resources are pooled from a diverse array of contributors, including independent data centers and cryptocurrency miners. These GPUs, often underutilized, are connected through a decentralized framework that allows them to operate as a cohesive network. NetMind Power, NetMind’s decentralized GPU network, leverages technologies like asynchronous gradient updates and efficient data segmentation to provide powerful, scalable AI computing resources at a fraction of the cost typically associated with cloud-based solutions.
FE: If GPUs are a scarce resource for some, how do you find idle capacity for your network?
KZ: Big Tech is in an arms race to own as much compute power as possible - this allows them to price the GPU and AI services however they want. At the scale that smaller AI firms operate, the scarcity of GPUs is not just about availability but also about the efficient management of existing resources. While Big Tech companies often rely on scale to maintain a competitive edge, our approach focuses on optimizing the use of these resources in real-time. By leveraging Web3, we gain precise insights into where GPU resources are idle and where there is corresponding demand. Users are able to use NetMind Power and our token $NMT to allocate computing power across our decentralized network. By efficiently matching idle GPUs with demand, we create a more equitable and effective ecosystem, lowering costs for all participants.
FE: What kinds of clients want NetMind Power capacity and what are they using it for?
KZ: NetMind.AI’s decentralized GPU clusters are supported by a diverse range of contributors. Our users include smaller tech companies, academic researchers, and independent developers who leverage our clusters for various AI and machine learning projects. These projects range from training and fine-tuning large language models to inferencing LLMs, with their duration varying.
FE: How much do you charge them for what they use?
KZ: At NetMind.AI, our pricing is designed to be competitive and accessible, particularly for smaller companies and independent developers. We offer flexible pay-per-use models without the need for long-term contracts, making it easier for users to scale their AI projects without significant upfront costs.