FPGA-Accelerated Non-Volatile Memory Access

  • Conference:

    GI Operating Systems SIG Autumn Meeting 2022

  • Place:

    Erlangen, Germany

  • Date:

    September 19 - 20, 2022

  • Author:

    Yussuf Khalil, Thorsten Gröninger, Lukas Werling, Frank Bellosa

  • Speaker:

    Yussuf Khalil

  • Intel recently introcuded Optane Persistent Memory (PMem), a byte-addressable non-volatile memory that is attached directly to the processor’s memory bus and thereby allows access with traditional load/store instructions. This model enables very low latency, but can have severe performance deficits in practice. In particular, synchronous write accesses can lead to CPU stalls and leave less CPU time for other storage processing tasks (e.g., file systems, compression, encryption) and even completely unrelated CPU-bound processes. We propose mediating access to PMem using an FPGA-based PCIe device. This way, we can preserve low latencies while also freeing the CPU from certain storage processing tasks. Notably, in our system setup, the Optane memory is connected directly to the FPGA instead of the CPU’s memory controller. In the future, we plan to move away from PCIe in favor of the upcoming CXL.mem protocol. As a first step, we implement asynchronous copy offloading, which is a mechanism for overcoming performance issues from write stalls. Our design features SR-IOV support and a custom low-latency MMU tailored for the typical large sizes of Optane modules. The accompanying Linux driver provides user applications with a memcpy()-style interface that allows for parallelized submission of copy tasks without runtime overhead for locks or system calls.