Prof. Dr. Frank Bellosa

Abstract:

CPU performance is increasingly limited by thermal dissipation, and soon aggressive power management will be beneficial for performance. Especially, large parts of the chip (including the caches) will be frequently power-gated in order to reduce leakage power. Therefore, the cache content is lost whenever the CPU is
idle, which causes a performance loss when execution is resumed, due to the high number of cache misses when the working set is fetched from external memory. In a server system, the first network request during this period suffers from increased response time. For common workloads, we measured as much as 34% response
time overhead due to cold caches.

In this thesis, we present a technique to reduce this overhead by preheating the caches in advance before the network request arrives at the server: Our design predicts the working set of the server application by analyzing the cache contents after similar requests have been processed. As soon as an estimate of the working set is available, a predictable network architecture starts to announce future incoming network packets to the server, which then loads the predicted working set into the last-level cache in anticipation of the packets. When the network packet arrives after this preheating step is complete, the server application can process the network request with warm caches and therefore with improved performance.

We evaluate our design with a proof-of-concept prototype based on Linux, on a system with ARM CPUs. Our experiments show that cache preheating can reduce the response time overhead caused by cold caches by an average of 80%. We demonstrate that our prototype does not cause significant runtime overhead, and we show that the time required to load the working set into the cache is low enough that it is possible to exploit the full potential of cache preheating on current server CPUs.

BibTex:

@mastersthesis{gottschlag16preheatedcaches,
  author = {Gottschlag, Mathias},
  title = {Reduced Response Time with Preheated Caches},
  type = {Master Thesis},
  year = 2016,
  month = may # "31",
  school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
}