IBM Research introduced the Artificial Intelligent Unit, the group’s first complete SoC for running and training deep learning models. It works faster and more efficiently than a general-purpose CPU, IBM claimed.
AIU is a prototype ASIC designed to support multiple formats and streamline AI workflows from image recognition to natural language processing, the company said in a blog this week. It includes 32 processing cores built on 5nm node technology with the entire chip containing 23 billion transistors.
In one boast, IBM said the AIU is “designed to be as easy to use as a graphics card. It can be plugged into any computer or server with a PCIe slot.”
The 32 cores of AIU are a scaled-up version of the AI accelerator already built into IBM’s Telum chip, which powers the latest IBM z16 system. (Telum’s transistors are 7 nm while the AIU runs on 5nm transistors.)
IBM said it will soon share news about when AIU will be released.
One of the most interesting details about AIU goes back to IBM’s technique of approximate computing, which is a simplified format for compute functions that cuts down the amount of number crunching needed to train and run an AI model. It means a computation is dropped from 32-bit floating point arithmetic to bit-formats that hold a quarter as much information.
This leaner bit approach reduces the time it takes to move data to and from memory. That means running an AI model can be far less memory intensive. IBM also said it saves energy by sending data directly from one compute engine to the next, a simpler layout that a multi-purpose CPU.
While it may seem counterintuitive, lower precision is OK, IBM noted: “An AI chip doesn’t have to be as ultra-precise as a CPU. We’re not calculating trajectories for landing a spacecraft on the moon or estimating the number of hairs on a cat. We’re making predictions and decisions that don’t require anything close to that granular resolution.”
The AIU comes out of IBM Research’s AI Hardware Center, launched in 2019 with the goal of improving AI hardware efficiency by 2.5 times each year. By 2029, IBM’s goal is to train and run AI models one thousand times faster than in 2019. Industrial-scale hardware will be needed to predict hurricanes or recessions, IBM said. “Our AIU takes us one step closer,” IBM said.