Rethinking Storage I/O for Hybrid NVMe and DAX Block Devices

  • Type:Master Thesis
  • Date:19.07.2024
  • Supervisor:

    Prof. Dr. Frank Bellosa
    Yussuf Khalil, Lukas Werling

  • Graduand:Daniel Habicht
  • Links:PDF
  • Abstract
    With recent advances in cache coherent interconnects like Compute Express Link (CXL), providing low-latency load/store semantics for device-attached memory becomes feasible in practice. Next to the expansion of main memory, this also opens up the possibility for a new type of storage product that has been proposed in the past: hybrid storage devices combine a byte-addressable persistent cache accessed through load/store semantics, with conventional, cost-efficient Flash for bulk storage. Such devices seem especially promising for providing memory-like storage performance on workloads with strong persistence requirements and high locality. Storage abstractions in operating systems, however, have been tailored to asynchronous block interfaces that were used for conventional storage devices over decades. While Direct-Access (DAX) abstractions found in operating systems like Linux promise to support byte-addressable storage (e.g., Optane persistent memory), these abstractions are insufficient for hybrid storage because they assume a different, incompatible device model.
    In this thesis, we rethink hybrid storage support from the ground up. We propose a persistence-aware page cache for seamlessly integrating hybrid storage devices into the modern I/O stack. Our design aims to provide direct access to storage and optimize performance-critical operations like synchronous writeback (i.e., fsync). With the limitations of the small-capacity cache in mind, we design a new user space API centered around fine-granular control over mappings of cache memory. While out-of-scope for this thesis, our design provides the foundation for building towards transparent use of hybrid storage devices. We implement our approach in the Linux kernel.
    Our evaluation on micro benchmarks shows up to 37 × higher throughput for synchronous writeback. We are able to translate these improvements into up to 4.1× higher throughput on the key-value datastore Valkey while reducing the per-request CPU and energy overhead by 78% and 74%.

    BibTex:

    @masterthesis{Rethinking Storage I/O,
      author = {Daniel Habicht},
      title = {Rethinking Storage I/O for Hybrid
    NVMe and DAX Block Devices},
      type = {Master Thesis},
      year = 2024,
      month = july# "19",
      school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
      }