Prof. Dr. Frank Bellosa
Jens Kehne

Abstract:

Modern GPUs are immensely powerful, highly asynchronous computational accelerators usable for a wide range of applications. Unfortunately, operating systems still treat GPUs like simple devices with predictable response times [28] and are unable to enforce fair sharing of GPU computation time between multiple applications. Additionally, virtualization and cloud-computing became ubiquitous. Most modern GPUs, however, are closed black-box devices, which makes them difficult to virtualize. Previous work on both GPU scheduling and GPU virtualization exists, but does not target the intersection of both topics: GPU scheduling in virtualized environments.
In this thesis, we design, implement and evaluate two fundamentally different approaches for virtualized GPU scheduling. Our primary approach employs a central scheduler in the hypervisor in order to balance the GPU usage of applications over multiple levels of virtualization. We supply the central scheduler with paravirtual hints from the guest to allow the identification of guest tasks. By manipulating the virtual machine’s memory mappings of GPU channels, our scheduler is able to separately account and control the GPU usage of individual guest tasks.
Our second approach works fully decentralized and uses separate schedulers on the host and in each virtual machine. Each nested scheduler enforces fair GPU time sharing locally among child tasks.
Based on two existing systems for GPU scheduling and GPU virtualization, we implement a full prototype of the paravirtual approach and a proof of concept of the nested approach.
In the evaluation, we demonstrate our paravirtual prototype’s ability to enforce fairness both between virtual machines and between tasks running inside virtual machines at the same time. We measure an average scheduling overhead of only 2.17 %. Additionally, we demonstrate the feasibility of nested GPU scheduling with our decentralized approach. Our proof of concept is able to account and schedule GPU usage in the virtual machine without any help from the hypervisor.

BibTex:

@mastersthesis{Fleig,
  author = {Tobias Fleig},
  title = {Fair Scheduling of GPU computatio Time in virtualized environments},
  type = {Master Thesis},
  year = 2016,
  month = sept # "16",
  school = {Operating Systems Group, Karlsruhe Institute of Technology (KIT), Germany}
}