CXL moves forward as memory technology for AI

Compute Express Link (CXL), the technology for connecting memory, was among the themes at last week’s Future of Memory and Storage summit in Santa Clara.

CXL is an open standard for high-speed, processor-to-device and processor-to-memory connections. It’s designed for high-performance datacentre servers and enables processor modules to share memory.

It provides an interface based on the CXL.io protocol that uses the PCIe 5.0 physical layer and electrical interfaces to provide low-latency interconnect paths for memory access, communication between host processors and devices that need to share memory resources. This means memory on PC servers can be increased, even when all the memory slots (Dimm sockets) are full, as CXL uses the PCI express (PCIe) expansion bus. 

According to Samsung, CXL enables memory sharing and memory expansion seamlessly without the need to use additional technology infrastructure.

CXL Memory Modules (CMMs) aim to address the rapidly growing demand for high memory capacity in AI applications. Among its uses is to support the memory architectures required by large language AI models.

In a post on its community blog published in March, Intel noted that training large language models (LLMs) such as GPT-4, Llama 2 and PaLM 2 requires large memory capacity and compute capabilities.

The chipmaker discussed how CXL enables server manufacturers to lower the cost of hardware by utilising less expensive memory. On the blog, Intel said Micron provides 128 GB CXL modules using DDR4 memory, older-generation double data rate synchronous dynamic random-access memory chips.

CXL for server applications

Since 2022, Samsung has partnered with Red Hat to focus on the development and validation of open source software for existing and emerging memory and storage products. These include NVMe SSDs; CXL memory; computational memory/storage and fabrics.

In May, at the Red Hat Summit held in Denver, Samsung demonstrated its CMM-D1 memory model embedded in Red Hat Enterprise Linux 9.3, which it said enhances the performance of deep learning recommendation models (DLRM).

This represents the first demonstration of CXL infrastructure certified by Red Hat, for Red Hat Enterprise Linux.

The demo used Samsung’s open source Scalable Memory Development Kit (SMDK) memory interleaving software technology to improve memory access performance. Samsung said SMDK-equipped CMM-D memory, which supports CXL, allows developers to build high-performance AI models without having to make significant investments. According to Samsung, the technology accelerates data processing, AI learning and inferencing speeds.

Discussing the demo, Kyeong Sang Kim, Red Hat Korea general manager, said: “The optimisation of Samsung’s hardware for Red Hat’s software underscores the value of open source technology as an imperative when it comes to expanding next-generation memory solutions such as CMM-D.”

In the Red Hat ecosystem catalogue listing, Samsung describes how functionality in RHEL called memory tiering supports its CMM-D memory modules. According to Samsung, when seldom-accessed data is stored in local memory, memory tiering reduces the performance of frequently used, or “hot”, memory. The RHEL memory tiering function is designed to allocate hot memory to the local memory tier and infrequently-used memory to the CXL tier, allowing for data migration between tiers when necessary.