Sunnyvale, Calif., AI supercomputer company Cerebras states its next generation of waferscale AI chips can do double the efficiency of the previous generation while taking in the very same quantity of power. The Wafer Scale Engine 3 (WSE-3) consists of 4 trillion transistors, a more than 50 percent boost over the previous generation thanks to using more recent chipmaking innovation. The business states it will utilize the WSE-3 in a brand-new generation of AI computer systems, which are now being set up in a datacenter in Dallas to form a supercomputer efficient in 8 exaflops (8 billion drifting point operations per second). Independently, Cerebras has actually participated in a joint advancement arrangement with Qualcomm that intends to enhance a metric of cost and efficiency for AI reasoning 10-fold.
The business states the CS-3 can train neural network designs as much as 24-trillion specifications in size, more than 10 times the size these days’s biggest LLMs.
With WSE-3, Cerebras can keep its claim to producing the biggest single chip in the world. Square-shaped with 21.5 centimeters to a side, it utilizes almost a whole 300-millimeter wafer of silicon to make one chip. Chipmaking devices is usually restricted to producing silicon passes away of no greater than about 800 square millimeters. Chipmakers have actually started to leave that limitation by utilizing 3D combination and other sophisticated product packaging technology3D combination and other sophisticated product packaging innovation to integrate several passes away. Even in these systems, the transistor count is in the 10s of billions.
As typical, such a big chip includes some astonishing superlatives.
Transistors 4 trillion Square millimeters of silicon 46,225 AI cores 900,000 AI calculate 125 petaflops On chip memory 44 gigabytes Memory bandwidth 21 petabytes Network material bandwidth 214 petabits
You can see the result of Moore’s Law in the succession of WSE chips. The very first, debuting in 2019, was used TSMC’s 16-nanometer tech. For WSE-2, which showed up in 2021, Cerebras carried on to TSMC’s 7-nm procedure. WSE-3 is developed with the foundry giant’s 5-nm tech.
The variety of transistors has more than tripled because that very first megachip. What they’re being utilized for has actually likewise altered. The number of AI cores on the chip has actually considerably leveled off, as has the quantity of memory and the internal bandwidth. The enhancement in efficiency in terms of floating-point operations per 2nd (flops) has actually exceeded all other procedures.
CS-3 and the Condor Galaxy 3
The computer system developed around the brand-new AI chip, the CS-3, is created to train brand-new generations of huge big language designs, 10 times bigger than OpenAI’s GPT-4 and Google’s Gemini. The business states the CS-3 can train neural network designs as much as 24-trillion criteria in size, more than 10 times the size these days’s biggest LLMs, without turning to a set of software application techniques required by other computer systems. According to Cerebras, that indicates the software application required to train a one-trillion specification design on the CS-3 is as simple as training a one billion criterion design on GPUs.
