On September 27, 2024, the third GMIF2024 Innovation Summit successfully concluded at the Renaissance Shenzhen Bay Hotel. The summit brought together leaders and experts from the global storage industry to discuss the cutting-edge developments in storage technology in the AI era. Prof. Yimao Cai, Dean of the School of Integrated Circuits at Peking University, delivered a keynote speech on “The Evolution of Memory and In-Memory Computing Technologies in the AI Era”. Through an academic research lens and in relation to industry demands, Prof. Cai shed lights on the current state and future potential development paths of memory technology, and explained that in-memory computing technology is expected to take on an increasingly important role in the AI era. During the summit, the GMIF2024 Annual Awards were announced, and Dean Cai served as the presenter, honoring companies such as Arm, YMTC, CXMT, GigaDevice, and BIWIN Storage for their outstanding contributions to memory innovation.
Challenges and Opportunities in the AI Era
In his speech, Dean Cai pointed out that with the rapid development of artificial intelligence (AI), global memory and computing systems are facing unprecedented challenges. AI applications are thirsty for efficient computing and low power consumption solutions. However, as Moore’s Law gradually slows down and feature size scaling approaches its limits, system-level computing power and energy efficiency improvements have become focal issues in the post-Moore era. This challenge extends beyond chip R&D and manufacturing; it’s a shared issue that requires joint efforts from the entire upstream and downstream IC industry chain. Dean Cai highlighted that meeting high-performance computing demands under limited power consumption, especially in the context of “dual carbon” goals, is both a technical and economic issue requiring collaborative solutions from academia and industry.
In-Memory Computing: Key to Breaking Through Traditional Computing Architecture
Dean Cai elaborated on the development potential of In-Memory Computing (IMC) technology. By embedding computing capabilities within memory, IMC technology excels in reducing data transmission latency between processors and memory, further improving system performance and lowering energy consumption. He pointed out that research in in-memory computing technology has made significant progress, and shown broad application prospects and tremendous value, particularly in architectures based on DRAM, NAND Flash, and emerging memory technologies (such as RRAM).
Dean Cai emphasized that in-memory computing though faces some technical challenges at the moment, such as reliability issues with non-volatile memory, it has the potential to be dominant in future AI computing field through continuous research and innovation. Not only can IMC drive enhancement in AI inference but also provide efficient solutions for edge computing.
Multi-Tiered Heterogeneous Memory Architecture: The Future of AI Computing
As AI application scenarios trend toward diversification, a single memory type can no longer meet the demands of all computing tasks. Dean Cai believes that the storage architectures in the future are poised to move towards multi-tiered heterogeneous memory systems, where different memory types such as DRAM, NAND, and emerging memory technologies will work collaboratively through advanced packaging techniques to address AI computing needs in different scenarios.
He specifically mentioned that under the demands for high bandwidth and large capacity storage, the application of high-performance memory such as HBM has become a future industry trend. However, HBM alone is insufficient to meet all AI computing needs, especially in low-power, low-cost scenarios, where further improvements in overall energy efficiency are needed through reduced memory access frequency and optimized system architecture.
Future Prospects of Emerging Memory Technologies
Dean Cai also revealed that future breakthroughs in memory technology will not be limited to hardware upgrades solely. Introducing new computing paradigms such as brain-like computing, quantum computing, and photonic computing will become frontier directions for memory technology innovation in the AI era. These technologies have the potential to significantly improve AI computing efficiency and energy effectiveness through new information encoding methods.
Integration of Memory and AI: Driving Industry Transformation
In the conclusion of the keynote, Dean Cai proclaimed that the advancement of memory technology in the AI era is not only dependent on breakthroughs in capacity and speed but also on progresses in energy consumption and computing power. The applications of in-memory computing technology, multi-tiered heterogeneous memory architecture, and emerging memory technologies will provide strong support for the future development of AI computing. Prof. Cai called for collaboration between academia and industry to jointly promote innovation and progress in storage technology to better address the risks and challenges of the intelligent era.
Conclusion
With the rapid development of AI technology, innovation in memory and computing architecture has become an important topic in the field of information technology. Looking forward, the widespread application of multi-tiered heterogeneous memory architecture and in-memory computing technology will provide more efficient solutions for large-scale AI computing. The continuous exploration and research in this field by the School of Integrated Circuits at Peking University will also contribute to promoting global storage technology innovation, driving the integrated circuit industry toward a more intelligent and efficient future.
About School of Integrated Circuits, Peking University
The School of Integrated Circuits at Peking University traces its origins to the “Five-School Joint Semiconductor Specialization”, founded by Mr. Huang Kun in the 1950s. It is a pioneer in semiconductor science and technology research and talent cultivation in China. Established in 2021, the school boasts multiple national and provincial innovation research platforms, as well as international cooperation platforms, providing a world-class cutting-edge research environment. The school has been recognized with several national innovation team titles and has achieved a number of internationally influential outcomes. Focusing on academic frontiers and industry needs, the school has become a hub for technological innovation and talent cultivation in integrated circuits. It is a flagship of Peking University’s new engineering disciplines, making significant contributions to cultivating innovative talents, advancing technological innovation, and serving the country’s major strategic needs.