Digital Newsletter
Each week our editor Phil Alsop rounds up the most popular articles, videos and expert opinions. We compile this into a Digital Newsletter and send it straight to your inbox every week.
Digital Magazines
We'll let you know each time a new edition of Data Centre Solutions is released so that you're always kept up-to-date with the latest and greatest news and press releases.
Video Magazines
The Data Centre Solutions Video magazine contains the latest Zoom interviews with experts in the industry.
In the quest to enhance the efficiency of the burgeoning generative AI sector, Korean researchers have taken significant strides with the development of a novel NPU (Neural Processing Unit) core technology. As the demands of powerful AI models like OpenAI's ChatGPT-4 and Google's Gemini 2.5 continue to swell in terms of memory requirements, such advancements are crucial.
Professor Jongse Park and his team from KAIST School of Computing, in partnership with HyperAccel Inc., have introduced an NPU core that stands out not just for its impressive performance metrics but also for its energy efficiency. Their efforts are set to be showcased at the '2025 International Symposium on Computer Architecture (ISCA 2025)', a testament to its groundbreaking nature.
The core aim of the research revolves around optimising performance for large-scale generative AI services, achieved by lightweighting the inference process without sacrificing accuracy. The innovation is acknowledged for its harmonised design of AI semiconductors and system software, integral to AI infrastructure.
Traditionally, GPU-based AI setups demand multiple units to satisfy memory bandwidth and capacity needs. However, the NPU technology introduced here employs KV cache quantisation, revolutionising resource usage. Through this, fewer devices are needed, cutting costs in building and operating generative AI platforms.
Key to the hardware architecture is an adaptation that retains compatibility with existing NPUs while integrating advanced quantisation algorithms and page-level memory management. These innovations ensure maximal utility of available memory resources, optimising operations and further cutting power requirements.
With over 60% performance enhancement against traditional GPUs while using 44% less power, this achievement underscores the potential of NPUs in architecting robust and sustainable AI solutions. As AI technology continues its rapid ascent, the fruits of this research signify a pivotal turning point in striving toward state-of-the-art AI ecosystems.
