IBM teams with industry partners to bring energy-efficient AI hardware to hybrid cloud environments

The next generation of computing will define how we respond to crises, by accelerating the analysis of problems and the synthesis of solutions to them. The ability to make advances in AI, in particular, will empower collaborative scientific communities and bring automation and virtually unlimited computing resources to every facet of the scientific process. With the help of AI and the computers that power it, scientists will be able to accelerate and scale scientific discovery at a pace never before seen.

AI’s unprecedented demand for data, power and system resources poses the greatest challenge to realizing this optimistic vision of the future. To meet that demand, we’re developing a new class of inherently energy-efficient AI hardware accelerators that will increase compute power by orders of magnitude, in hybrid cloud environments, without the demand for increased energy.

At today’s IEEE CAS/EDS AI Compute Symposium, IBM Research is introducing new technology and partnerships that will enable companies to dynamically run massive AI workloads in hybrid clouds:

• IBM Research is collaborating with Red Hat, one of the newest members of the IBM Research AI Hardware Center, to make IBM Digital AI Cores compatible with Red Hat OpenShift and its ecosystem.
• We are making our toolkit for Analog AI Cores open source.
• The AI Hardware Center will collaborate with Synopsys, Inc. to address the challenges of developing new AI chip architectures.
• We are investing in infrastructure to accelerate new chip packaging development to eliminate memory bandwidth bottlenecks.

Red Hat Joins AI Hardware Center to help bring IBM Digital AI Cores to Red Hat OpenShift

Our AI Hardware Center, a global research hub headquartered in Albany, New York, plays a central role in accelerating the development of hardware purpose built for AI. Through its involvement in the AI Hardware Center, Red Hat is collaborating with IBM to build compatibility between IBM Digital AI Cores and Red Hat OpenShift, the industry’s most comprehensive enterprise Kubernetes platform. Digital AI Cores serve as accelerators, using custom architecture, software and algorithms to transform existing semiconductor technologies to apply reduced precision formats to speed computation and decrease power consumption while maintaining model accuracy.

Red Hat is collaborating with IBM’s AI hardware development stream and working to enable AI hardware accelerator deployment across hybrid cloud infrastructure: multi-cloud, private cloud, on-premise and edge. These accelerators can be used to build and deploy neural network models for applications that perform a spectrum of AI tasks including speech recognition, natural language processing and computer vision. The integration of accelerators based on IBM Digital AI Cores with Red Hat OpenShift enables the accelerators to be deployed and managed as part of a hybrid infrastructure.

As IBM is developing AI hardware, we can work on the software integration in parallel with Red Hat. This has two large benefits: The software and hardware can be ready at the same time (shortening the overall development cycle by months if not years), and the opportunity exists for a better overall solution.

Open Source Analog AI Cores

In traditional hardware architecture, computation and memory are segregated in different locations. Information is moved back and forth between computation and memory units every time an operation is performed, creating a limitation called the The von Neumann bottleneck can also be thought of as the limited throughput between CPU and memory, as compared to the total amount of memory.von Neumann bottleneck . We are developing analog AI that could provide significant performance improvements and energy efficiency by combining compute and memory in a single device, significantly alleviating this bottleneck.

IBM Research is exploring ways to create more powerful, yet efficient, AI systems with digital and analog cores. We’re now releasing our Analog Hardware Acceleration Kit as an open source Python toolkit that enables a larger community of developers to test the possibilities of using in-memory computing devices in the context of AI.

The Analog Hardware Acceleration Kit has two main components: PyTorch is an open source machine learning library based on the Torch library, a scientific computing framework with wide support for machine learning algorithms. PyTorch is used for developing AI applications such as computer vision and natural language processing.PyTorch integration and an analog devices simulator. The analog kit allows AI practitioners to evaluate analog AI technologies while allowing for customizing a wide range of analog device configurations, and the ability to modulate device material parameters. The goal is to enlist the AI community to assess the capabilities of analog AI hardware, to be part of a community that tailors models to extract the full potential of the hardware, and to invent new AI applications that can exploit the potential of this breakthrough technology.

Expanding Partnerships Around the AI Hardware Center

The IBM Research AI Hardware Center’s goal is to be the nucleus of a new ecosystem of research and commercial partners collaborating with IBM to further accelerate the development of AI-optimized hardware innovations. To date, we’ve made great progress in these efforts and expanded the Center’s partnerships to a total of 14 members.

This includes efforts with Synopsys, a leader in electronic design automation software and emulation and prototyping solutions. Synopsys also develops IP blocks for use in the high-performance silicon chips and secure software applications driving advancements in AI.

Going forward, Synopsys will serve as lead electronic design automation (EDA) partner for IBM’s AI Hardware Center, helping drive IBM’s vision of 1,000 times improvement in AI compute performance in the coming decade. We began collaborating last year and have already begun to achieve silicon verification and demonstrable performance improvements.

Heterogenous Integration

AI requires a lot of interconnect bandwidth connectivity to take advantage of increases in computing power. IBM and NY Creates are investing in a new cleanroom facility on the campus of AI Center member, SUNY-Poly, in Albany, New York, that will focus on advanced packaging, also called “heterogeneous integration,” to improve memory proximity and interconnect capabilities. This work will also help ensure that, as our new compute cores are developed, the memory bandwidth is increased in tandem. Otherwise, the compute units can remain idle waiting for data, leading to unbalanced system performance.

Our heterogeneous integration work focuses on three approaches to address the bandwidth issue:

1. advanced, fine pitch laminate technology;
2. implementation of silicon bridges embedded in the laminate to provide fine pitch interconnect between AI chips;
3. full 3D integration through the stacking of memory with the AI chip.

These approaches involve materials, processing and design challenges requiring a range of competencies to address them. For that reason, we recently launched a new, advanced packaging center in Albany, with six new members focused on advanced packaging.

As AI empowers society to extend scientific exploration, we will increasingly be confronted with large data processing workloads that demand breakthroughs in processing power, memory and bandwidth. Working with Red Hat, Synopsys and other partners, our advancements in AI hardware and hybrid cloud management software integration will enable models and methods that will forever change the way we solve problems.