GI Operating Systems SIG Autumn Meeting 2022

Erlangen, Germany

September 19 - 20, 2022

Yussuf Khalil, Thorsten Gröninger, Lukas Werling, Frank Bellosa

Yussuf Khalil

Intel recently introcuded Optane Persistent Memory (PMem), a byte-addressable non-volatile memory that is attached directly to the processor’s memory bus and thereby allows access with traditional load/store instructions. This model enables very low latency, but can have severe performance deficits in practice. In particular, synchronous write accesses can lead to CPU stalls and leave less CPU time for other storage processing tasks (e.g., file systems, compression, encryption) and even completely unrelated CPU-bound processes. We propose mediating access to PMem using an FPGA-based PCIe device. This way, we can preserve low latencies while also freeing the CPU from certain storage processing tasks. Notably, in our system setup, the Optane memory is connected directly to the FPGA instead of the CPU’s memory controller. In the future, we plan to move away from PCIe in favor of the upcoming CXL.mem protocol. As a first step, we implement asynchronous copy offloading, which is a mechanism for overcoming performance issues from write stalls. Our design features SR-IOV support and a custom low-latency MMU tailored for the typical large sizes of Optane modules. The accompanying Linux driver provides user applications with a memcpy()-style interface that allows for parallelized submission of copy tasks without runtime overhead for locks or system calls.

• Slides