Prof. Dr. Frank Bellosa
Lukas Werling
 

Abstract

Intel Optane Persistent Memory (PMem) is a recent non-volatile memory technology that provides higher integration density than current DRAM-based memories. PMem oers traditional load/store semantics with byte addressability and may be used as additional main memory complementary to DDR4 modules. Access performance is, however, severely worse than with DDR4 memory to the point where CPU cores may become stalled for prolonged times while waiting for memory operations to complete. Previous works have shown that this behavior can negatively impact the runtime performance of unrelated processes running concurrently on the same system. Asynchronous copy ooading has been proposed as a mechanism to overcome the performance implications of parallel accesses to PMem. However, previous implementations based on Intel I/OAT have been unsuccessful as I/OAT hardware is incapable of saturating Optane bandwidth. In this thesis, we present the design of a FPGA-based PCIe accelerator device for asynchronous copy ooading. To reduce the latency impact of data transfers via the PCIe bus, we connect the PMem modules directly to the FPGA instead of the CPU and design a custom MMU optimized specically for typical PMem use cases. Based on SR-IOV, we implement a multiplexing scheme that enables lock-free parallel command submission while maintaining proper process isolation without requiring kernel mediation. We weave the pieces together to oer userspace processes an interface that provides memcpy() semantics with asynchronous completion. We show that our design is capable of saturating PMem’s bandwidth while signicantly reducing the CPU time spent for memory operations by up to 98.8 %. Regarding latency, we observe a slowdown by up to 100×. Note: This work makes use of proprietary technology that was provided by Intel Corporation under the terms of a non-disclosure agreement. The respective passages are therefore censored in the public version of this document.

BibTex:

@masterthesis{khalil22FPGA,
  author = {Yussuf Khalil},
  title = {FPGA-Accelerated Non-Volatile Memory Access},
  type = {Master Thesis},
  year = 2022,
  month = oct # "27",
  school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
  }