Prof. Dr. Frank Bellosa

Daniel Habicht

Yussuf Khalil

Abstract
Compute Express Link (CXL) is a new interconnect protocol that enables adding memory to a host besides native DRAM. However, the latencies of CXL devices are not on par with DRAM directly attached to the system memory bus, especially not in overload situations, i.e., when memory requests are sent out faster to the CXL device than it can handle them, resulting in exponentially growing queueing delay. In this thesis, we aim to reduce CXL access latencies caused by queueing delay, thereby also reducing stall cycles of the CPU waiting for CXL memory requests, increasing the throughput of the CPU. For this, we implement hardware memory access counters on an FPGA-based CXL device and use them in combination with Intel Processor Event Based Sampling (PEBS) to get a per-process estimate of the CXL bandwidth usage. To avoid overloading the CXL device, we limit the CXL bandwidth of processes by reducing their CPU time. We implement two approaches to enforce CPU limits: A custom scheduler using the eBPF-based sched_ext interface of the kernel, and a userspace program that uses Linux control groups (cgroups) to limit CPU resource usage. We evaluate how both approaches perform in keeping the latency low in overload situations while still utilizing most of the bandwidth of the CXL device, and how the CPU throughput is affected. As a result, we achieve about 30 % lower CXL memory access latencies under load and a reduction of the CPU cost for reads of about 40 % compared to an unmodified Linux.

BibTex:

@masterthesis{kaemmerer25HardwareassistedMemoryBandwidthManagement,
  author = {Linus K{\"a}mmerer},
  title = {Hardware-Assisted Memory Bandwidth Management for CXL-based Memory Devices},
  type = {Master Thesis},
  year = 2025,
  month = oct# "29",
  school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
}