Prof. Dr. Frank Bellosa
Marc Rittinghaus

Abstract
In order to speed up record and replay of full virtual machines systems like SimuBoost employ hardware-assisted virtualization to limit the impact of the recording process on the virtual system’s performance. Like all record and replay systems this requires accurate logging of non-deterministic events that occur during the recording and injection of these events during the replay. The deterministic instructions between these events are being replayed directly.

However, current record and replay systems do not account for non-deterministic events that can be introduced by the x86 memory management unit (MMU). The unpredictable behavior of the translation lookaside buffer (TLB) and its potential incoherency with the page tables in RAM can cause theMMU to either set accessed and dirty bits in the page tables or skip this step depending on the TLB’s contents. When the recorded system reads the state of these bits, the replay can diverge from the recorded execution if the TLB’s contents or its behavior differ between the two, which is almost unavoidable.

We propose a way to record the changes the MMU makes to the status bits in the guest’s page tables and implement it as part of the SimuBoost system. We achieve this by protecting the page tables in using the extended page table (EPT) mechanism on modern Intel CPUs and emulating these instructions using the emulation facilities for x86 code within the KVM Linux module. We implement support for replaying these events in the QEMU fork QSIMU that SimuBoost utilizes for heterogeneous replay.

Our results show that this approach can be a feasible solution. While we observe a 350% slowdown during recording for worst-case micro-benchmarks, the impact on some real-world applications is only 50% and CPU intensive workloads as well as the replay of any of the workload are not affected at all. This system is also able
to successfully replay workloads the existing system consistently failed to replay before.

BibTex:

@masterthesis{morbach18SimuBoost,
  author = {Benedikt Morbach},
  title = {Accurate Record and Replay of x86 MMU Behavior for SimuBoost},
  type = {Master Thesis},
  year = 2018,
  month = sept # "13",
  school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
  }