Artificial intelligence (AI) has revolutionized various industries, from speech recognition to generative AI used in creating video and images. However, one of the challenges faced by AI systems is the increasing cost and energy consumption associated with training and operation. To address this issue, IBM has developed an AI chip that is significantly more energy efficient than conventional microchips, particularly in the area of speech recognition.
The new AI chip, developed by IBM researchers, is based on a 14-nanometer analog design and incorporates 35 million phase-change memory cells. This chip offers promising solutions for improving the energy efficiency of AI systems, including large language models like ChatGPT and generative AI used in the creation of art and media.
The Need for Energy Efficiency in AI Systems
The accuracy of automated transcription, a task greatly improved by AI, has made significant progress over the past decade. However, the hardware required to train and operate AI systems is becoming increasingly expensive and energy-hungry. For instance, OpenAI spent a staggering $4.6 million to run 9,200 GPUs for two weeks in order to train its state-of-the-art AI GPT-3.
One major obstacle in achieving energy efficiency is the substantial energy and time lost in transferring large amounts of data between processors and memory. This energy dissipation can be several orders of magnitude higher than the actual computational requirements. It is crucial to develop innovative hardware solutions that reduce energy consumption and improve overall efficiency.
Analog AI and Phase-Change Memory
Analog AI is a concept that mimics the way biological neurons compute and store data. It seeks to perform computations directly within memory, reducing the need for data transfer between processors and memory. Previous simulations from IBM suggested that analog AI could be significantly more energy efficient compared to traditional GPUs used in AI applications.
In their recent study, IBM researchers focused on phase-change memory, a technology that relies on a material capable of switching between amorphous and crystalline phases when subjected to electrical pulses. This allows the memory to encode results of multiply-accumulate (MAC) operations using just a few resistors or capacitors instead of hundreds or thousands of transistors required by conventional approaches.
Energy Efficiency and Performance
The IBM researchers developed a 14-nanometer microchip with 35 million phase-change memory cells distributed across 34 tiles. The chip demonstrated remarkable energy efficiency, capable of performing up to 12.4 trillion operations per second per watt. This level of efficiency far surpasses that of the most powerful CPUs and GPUs currently available in the market.
To evaluate the chip’s performance, the researchers used two speech-recognition neural-network programs. The results showed that their device performed as accurately as neural networks run on conventional hardware while being significantly faster and more energy efficient. For instance, when analyzing Google Speech Commands, the chip performed the task seven times faster, and for Librispeech, the chip was 14 times more energy efficient.
Implications for Large Language Models and Generative AI
Intel Labs’ Hechen Wang highlights the potential impact of this new microchip on large language models (LLMs) such as ChatGPT, which utilize transformer-based neural networks. LLMs have gained prominence in chatbot applications, passing exams, generating content, and answering interview questions. The energy-efficient chip can substantially reduce the power consumption and cost associated with running LLMs and generative AI systems.
However, LLMs and generative AI have also faced criticism due to their flaws and ethical implications. ChatGPT, for example, has been known to produce error-ridden articles, and generative AI raises concerns about intellectual property rights. It is important to address these challenges alongside technical advancements to ensure responsible and ethical use of AI technology.
Conclusion
IBM’s energy-efficient AI chip represents a significant step towards improving the energy efficiency and performance of AI systems. By leveraging phase-change memory and analog AI concepts, this chip demonstrates remarkable potential in reducing the energy consumption and cost associated with running large language models and generative AI applications. However, further advancements and considerations are necessary to overcome existing limitations and address ethical concerns in AI development and deployment.
The IBM scientists detailed their findings online 23 August in the journal Nature.
