Founded: 2016
Headquarters: Sunnyvale, California
Cerebras Systems is a developer of computing chips designed for the singular purpose of accelerating AI. Cerebras builds computer systems for complex artificial intelligence deep learning applications. With a team of pioneering computer architects, computer scientists, deep learning researchers, and engineers of all types, Cerebras creates chips that are the size of wafers, which are trays that would usually have several chips on it, to train AI models to accomplish tasks faster. The Company’s flagship product, the CS-2 system, which is powered by the world’s largest processor – the 850,000 core Cerebras WSE-2, enables customers to accelerate their deep learning work by orders of magnitude over graphics processing units.
Key Facts
“At Cerebras Systems, our goal is to revolutionize computing. It’s thrilling to see some of the most respected supercomputing centers around the world deploying our CS-2 system to accelerate their AI workloads and achieving incredible scientific breakthroughs in climate research, precision medicine, computational fluid dynamics and more.” said Andrew Feldman, CEO and Co-Founder of Cerebras Systems.
With enterprises in all sectors realizing the importance of AI, adoption is increasing. However, most AI projects fail due to not being cost effective.
According to recent surveys, business leaders have put the failure rate of AI projects between 83% and 92% due to AI not being cost efficient. “As an industry, we’re worse than gambling in terms of producing financial returns,” Aible CEO Arijit Sengupta says.
"AI will be in every electronic product we buy and sell and use from toys to automobiles. And the ability to make AI useful depends on really good engineers, really good AI modelers, data scientists, as well as the hardware. But all that will be for naught if you're not running it on efficient hardware because you can't afford it. So this is all about making AI pervasive and affordable and – people overuse the term – but democratizing AI is still a goal. It's not where we are, but it's gonna take a lot of work to get there,” according to the Founder of Cambrian AI Research.
Cerebras Systems’ main offering is a chip, The Wafer-Scale Engine (WSE). It is a revolutionary central processor for the Company’s deep learning computer system. The second-generation WSE (WSE-2) powers the CS-2 system: it is the largest computer chip ever built and the fastest AI processor on Earth.
Unlike legacy, general-purpose processors, the WSE was built from the ground up to accelerate deep learning.
The WSE-2’s main competitive advantages are: (1) Massive high bandwidth on-chip memory; (2) Speed (Faster than a traditional cluster could possibly achieve); (3) Cost and power efficiency (One chip is equivalent to a cluster of legacy machines); (4) Ease of programing; (5) Radically reduced programming complexity; and (6) Reduced wall-clock compute time, and time to solution
Cerebras’ WSE-2 gives unprecedented levels of computation, memory and interconnect bandwidth on a single, wafer-scale piece of silicon. Further optimizations by sparsity harvesting allow the computation capabilities to be maximized. The outcome is huge performance in an integrated chip without bottlenecks, in which every node is programmable and independent of others. With this revolutionary approach to AI, companies can reduce the cost of curiosity.
The net result of the Company’s innovation to date is unmatched utilization, performance levels, and scaling properties that were previously unthinkable.
Cerebras offers a revolutionary AI infrastructure, with its CS-2 system. The CS-2 is designed from the ground up to power, cool, and deliver data to the revolutionary WSE-2 processor so that it can deliver unparalleled performance to users. The package is easy to deploy, operate, and maintain in client datacenters today. This means no compromises and peak performance with no large-scale cluster deployment complexity for customers.
Large-scale data centers use mass amounts of computers, while Cerebras offers far less supercomputers for the same output. This means customers can deploy datacenter-scale AI computers to unlock world leading innovation in just a few days or weeks – delivering greater performance in a space- and power-efficient package built for the job.
Cerebras also offers the Cerebras Software Platform, CSoft. The Cerebras Software Development Kit allows researchers to extend the platform and develop custom kernels – empowering them to push the limits of AI and HPC innovation. The Cerebras Machine Learning Software integrates with the popular machine learning frameworks TensorFlow and PyTorch, so researchers can effortlessly bring their models to the CS-2 system.
Cerebras also offers Cloud Cirrascale. Cerebras Cloud provides access to a blazing fast AI solution, right at customers fingertips. It provides access to the latest Cerebras technology hosted at Cirrascale Cloud Services, a specialist in deep learning infrastructure. This joint solution delivers the model training performance of many processors in a single instance, so users can develop new neural network architectures, Machine Learning methods, and algorithms that were previously impractical – all without worrying about infrastructure management.
Cerebras can double the amount of computing its chips do for double the power, unlike current systems that need more than twice as much power to double their computing capacity. Current AI systems use tens of megawatts of power over months and use the equivalent of a small city’s power to train models – Cerebras allows the same computer power with far less energy at faster speeds.
Cerebras allows researchers and organizations with tiny budgets — in the tens of thousands of dollars range — to access AI training tools that were previously only available to much larger organizations with lots of money.
In August 2022, Cerebras set a record as the only single system capable of training Transformer-Style Natural Language AI Models with 20x longer sequences. This new capability is expected to lead to breakthroughs in natural language processing. By providing vastly more context to the understanding of a given word, phrase or strand of DNA, the long sequence length capability enables Natural Language models a much finer-grained understanding and better predictive accuracy.
In June 2022, Cerebras set another world record for the largest AI model trained on a single device. This is important because with natural language processing, larger models trained with large datasets are shown to be more accurate.
In August 2022, Cerebras Systems announced its continued global expansion with the opening of a new India office in Bangalore, India. Led by industry veteran Lakshmi Ramachandran, the new engineering office will focus on accelerating R&D efforts and supporting local customers.
In August 2022, Cerebras’ WSE Chip was accepted into the Computer History Museum world-renown collection that contains technology of past, present and future. President & CEO of the Computer History Museum said, “Cerebras Systems’ extraordinary accomplishment of inventing the world’s first and only wafer-scale processor marks a key milestone in the history of computing, and we’ve only just begun to see the incredible impact that Cerebras Systems and their customers are having across AI for drug discovery, climate change, cancer research, and so much more.”
Cerebras Systems’ AI technology recently powered a study by Argonne National Laboratory that studied how COVID-19 works, and the study that was nominated as a Gordon Bell Special Prize finalist.
In November 2022, Cerebras unveiled Andromeda, a 13.5 million core AI supercomputer, now available and being used for commercial and academic work. Built with a cluster of 16 Cerebras CS-2 systems and leveraging Cerebras MemoryX and SwarmX technologies, Andromeda delivers more than 1 Exaflop of AI compute and 120 Petaflops of dense compute at 16-bit half precision. It is the only AI supercomputer to ever demonstrate near-perfect linear scaling on large language model workloads relying on simple data parallelism alone.
In November 2022, Cerebras and Jasper, the category-leading AI content platform, announced a partnership to accelerate adoption and improve the accuracy of generative AI across enterprise and consumer applications. Using Cerebras’ Andromeda AI supercomputer, Jasper will train its profoundly computationally intensive models in a fraction of the time and extend the reach of generative AI models to the masses.
In February 2023, the National Energy Technology Laboratory and Pittsburgh Supercomputing Center pioneered the first ever computational fluid dynamics simulation on Cerebras' wafer-scale engine. The simulation is expected to run several hundred times faster than what is possible on traditional distributed computers, as has been previously demonstrated with similar workloads.
In February 2023, Cerebras launched Cerebras AI Model Studio. With the Cerebras AI Model Studio, users now have access to various large language models, including GPT-J (6B), GPT-NeoX (20B), and CodeGen (350M to 16B), with more models and checkpoints coming soon. Whether users are breaking new ground and training a model from scratch, fine-tuning a generic pre-trained model to extract better domain-specific performance, or continuously training a model from an existing checkpoint, Cerebras now has them covered. With this launch, users can fine-tune models like GPT-J, GPT-NeoX, and CodeGen faster, cheaper, and easier than with competing solutions.
In March 2023, Cerebras announced the release to the open source community of Cerebras-GPT, a family of seven GPT models ranging from 111 million to 13 billion parameters. Trained using the Chinchilla formula, these models provide the highest accuracy for a given compute budget. Cerebras-GPT has faster training times, lower training costs, and consumes less energy than any publicly available model to date.
In July 2023, Cerebras Systems and G42, the UAE-based technology holding group, today announced Condor Galaxy, a network of nine interconnected supercomputers, offering a new approach to AI compute that promises to significantly reduce AI model training time. The first AI supercomputer on this network, Condor Galaxy 1 (CG-1), has 4 exaFLOPs and 54 million cores. Cerebras and G42 are planning to deploy two more such supercomputers, CG-2 and CG-3, in the U.S. in early 2024. With a planned capacity of 36 exaFLOPs in total, this unprecedented supercomputing network will revolutionize the advancement of AI globally. “Collaborating with Cerebras to rapidly deliver the world’s fastest AI training supercomputer and laying the foundation for interconnecting a constellation of these supercomputers across the world has been enormously exciting. This partnership brings together Cerebras’ extraordinary compute capabilities, together with G42’s multi-industry AI expertise. G42 and Cerebras’ shared vision is that Condor Galaxy will be used to address society’s most pressing challenges across healthcare, energy, climate action and more,” said Talal Alkaissi, CEO of G42 Cloud, a subsidiary of G42.