Design and Modelling of Cloud-Based Burst Buffers
Authors: Tianqi Xu (Tokyo Institute of Technology), Kento Sato (Lawrence Livermore National Laboratory), Satoshi Matsuoka (Tokyo Institute of Technology)
Abstract: With the growing of data size, public clouds have been gathering more and more interests due to their capabilities of public big data processing. However, applications running on cloud are suffering from low I/O bandwidth as well as loose consistency model in shared cloud storages. As previous work, we have proposed cloud-based burst buffers (CloudBB) to accelerate the I/O and strengthen the consistency while using shared cloud storages. In this work, we introduce the performance models to predict the performance and help users to determine the optimal configuration while using our system. We focus on two aspects: execution time and cost. Our model predicts the optimal configuration according to characteristics of applications and execution environment. We validate our model using a real HPC application on a real public cloud, Amazon EC2/S3. The results show that our model can predict the performance and help users to determine the optimal configuration.
Two-page extended abstract: pdf