SC15 Austin, TX

BurstFS: A Distributed Burst Buffer File System for Scientific Applications

Authors: Teng Wang (Florida State University), Kathryn Mohror (Lawrence Livermore National Laboratory), Adam Moody (Lawrence Livermore National Laboratory), Weikuan Yu (Florida State University), Kento Sato (Lawrence Livermore National Laboratory)

Abstract: Large-scale scientific applications on leadership computing facilities usually generate colossal amount of scientific datasets. These datasets impose a great challenge to the backend storage systems for timely data service. Recently, the idea of burst buffer was proposed as a promising storage solution to cope with the exploding data pressure. Node-local burst buffers, which locate on the individual compute node, have been already deployed on several HPC systems. Despite of their great potential for massive scalability, there is still a lack of software solution to leverage such storage architecture. In this study, we propose BurstFS, a distributed burst buffer file system, to exploit the node-local burst buffers, and provide scientific applications with efficient, scalable I/O service. Our initial evaluation demonstrated that BurstFS increased bandwidths by 1.83X and its performance scaled linearly with the number of involved processes.

Poster: pdf
Two-page extended abstract: pdf

Poster Index