Low-Latency Synchronous IO For OpenZFS Using Persistent Memory

  • Type:Master Thesis
  • Date:07.06.2021
  • Supervisor:

    Prof. Dr. Frank Bellosa
    Lukas Werling

  • Graduand:Christian Schwarz
  • Links:PDF
  • Abstract
    OpenZFS is a storage system that combines volume management and lesystem services. The ZFS Intent Log (ZIL) is ZFS’s mechanism for supporting synchronous IO semantics. To improve ZIL performance, ZFS allows for the conguration of a separate log device (SLOG) that it uses exclusively for ZIL allocations.
    Persistent memory (PMEM) is an emerging technology that provides low-latency memory-mapped byte-addressable persistent storage. The Linux kernel’s /dev/pmem pseudo block device allows existing block device consumers to benet from PMEM’s high throughput and low latency without modication.
    To explore the use of PMEM as a storage medium for the ZIL, we congure a ZFS storage pool that uses /dev/pmem as a SLOG. We nd that the current ZIL implementation (ZIL-LWB) exhibits signicantly higher latency and sub-par throughput compared to the raw PMEM hardware in a 4k synchronous random write workload. An analysis of wall clock time distribution among the ZFS components involved in this type of IO
    operation reveals that block-device-oriented abstractions and data structures account for the vast majority of the overall latency.
    Motivated by this observation, we propose a new type of ZIL called ZIL-PMEM that exclusively targets persistent memory to take advantage of its remarkable performance characteristics. We refactor ZFS to support dierent ZIL kinds at runtime, enabling coexistence of ZIL-LWB and ZIL-PMEM. ZIL-PMEM maintains the same crash consistency guarantees towards userspace as ZIL-LWB and uses the same checksum to ensure data integrity. We validate our core data structure through extensive unit testing as well as the upstream test suite and stress testing tool. Our implementation shows high speedups over ZIL-LWB, with a maximum of 8x in a single-threaded 4k synchronous random write workload on the same storage hardware.


      author = {Christian Schwarz},
      title = {Low-Latency Synchronous IO For OpenZFS Using Persistent Memory},
      type = {Master Thesis},
      year = 2021,
      month = jun # "07",
      school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}