Memory Management for CXL-Based Memory Architectures
While CXL provides the infrastructure for memory expansion, utilizing these resources efficiently remains challenging. Each memory expansion device possesses distinct characteristics determined by the CXL topology, the CXL controller, the memory controller, as well as the underlying memory cell technology. These characteristics include memory access latency and bandwidth, but also aspects such as energy consumption. Strict requirements regarding resource and bandwidth limits as well as tail latencies require careful software-based management when combining multiple types of memories with different characteristics.
The primary focus of this project is resource management for heterogeneous memory architectures to avoid CPU stalls due to overload situations and achieve a practical trade-off between throughput and tail latencies on a per-process basis. To achieve this goal, we need 1) insights into how much bandwidth each thread consumes per memory backend, 2) a resource allocation strategy that avoids overloading memory devices while ensuring fairness, and 3) a mechanism for enforcing resource limits.
While modern CPUs typically feature bandwidth monitoring capabilities, these either cannot attribute bandwidth to an individual thread or differentiate between different memory resources (e.g., local DRAM and CXL-attached DRAM). In order to provide our resource allocator with accurate, per-process bandwidth statistics, we explore the use of hardware-assisted bandwidth monitoring implemented on our FPGA-based in-house memory expander design.
correlate with the amount of CPU stalls. Loaded latency measured
with Intel memory latency checker (MLC).
As illustrated in Figure 1, saturating the memory bandwidth leads to an exponential increase in access latency and stall cycles. Therefore, the resource allocator must carefully assign bandwidth resources to each thread while respecting process priorities and tail latency requirements. This is especially important for CXL devices as they can typically be saturated by few cores and have a higher base latency.
To enforce bandwidth limits set by the resource allocator, we implement a bandwidth-aware scheduler based on Linux’s sched_ext infrastructure. Contrary to approaches that artificially delay memory accesses for limiting bandwidth, our scheduling-based approach can enforce bandwidth limits without slowing down the CPU. By avoiding situations that lead to oversaturated memory devices, our scheduler frees up CPU time that can then be reassigned to less memory-bound tasks.
Contact: Daniel Habicht