Modifying the File System for the GPU

Commonly, file systems are accessed using a POSIX-like interface that has traditionally been developed for CPU-side systems with system call support. While a POSIX-compatible interface is also required for compatibility and supported thus by our GPU-side system, the interface requires multiple round-trips between the file system driver and the application. To achieve high performance on GPUs we need to support parallel and asynchronous accesses, which is prohibited by the need for round-trips. Instead, we offer an an additional, asynchronous interface that bundles commonly used POSIX-style requests into a single operation: An example is loading a whole file into a memory buffer: Instead of opening the file, requesting the file size using the stat call, allocating a buffer accordingly, reading into the buffer and then closing the file, our interface can simply load a file directly into an automatically allocated buffer and only return the pointer to the buffer together with the buffers length to the application. This new interface also offers additional angles of optimizations: On classical file systems, the in-memory file system cache has no idea of the usage patterns of software. Therefore, the file system has to guess the lifetimes of in-cache data, leading to both higher cache usage and unnecessary rereads from disk. With the new interface, we can incorporate caching hints, e.g., that data is expected to be read only once or that the cache should retain the data. This reduces both cache usage and rereads, while increasing performance.

Offering a new interface poses challenges when porting incompatible existing software and educating developers to take advantage of the newly added features. However, we argue that for GPU4FS, both of these are less of an issue as compared to classical software:

• First, currently virtually no GPU-side file system software exists, so the code has to be developed freshly and can directly incorporate the new interface where appropiate.
• Second, most applications abstract the reading of a file into a subroutine, and programmers do not commonly think on that level of abstraction. Our new interface therefore only replaces the hidden implementation of said subroutine.

Running on a parallel computer like a GPU also requires some major changes in the file system design. Traditionally, file systems were built from mostly slim, deep trees to save on CPU cycles, instead opting to add frequent branch instructions and data fetches depending on earlier results from both memory and storage. On GPUs, branches and data fetches are much more costly than on CPU, however, compute is much cheaper. Therefore, while we still employ some tree-like structures to quickly find the required data, we design the trees to be flat to reduce the number of dependent fetches and instead load more bytes with a single fetch which can be quickly searched. Similarly, we employ wide array- and tree-like structures in our runtime file system cache management system.

Lastly, recent file systems added more compute-intensive features, e.g., checksumming to ensure data correctness or deduplication to reduce storage usage. Given the high parallel processing power of GPUs, we explore adding these features to our system and measure their performance impact.