Paper Info
Title
Tuning Object-Centric Data Management Systems for Large Scale Scientific Applications
Abstract
Efficient management of scientific data on high-performance computing (HPC) systems has been a challenge, as it often requires knowledge of various hardware and software components of the system, as well as tedious manual effort in optimizing parallel I/O for each application. This situation is exacerbated by the fact that storage systems on upcoming exascale supercomputers are equipped with an unprecedented level of complexity due to a deep storage and memory hierarchy with heterogeneous hardware and their management software. Simple and effective data management methods are critical for numerous scientific applications that are storing and analyzing massive amounts of data on HPC systems. Object-centric data management systems (ODMS) provide an easy-to-use interface, allow for massive scalability with relaxed consistency, and have been gaining popularity in the HPC community. However, tuning an ODMS to achieve its full potential on existing HPC systems with large-scale science use cases still remains a challenging task. In this paper, we explore and evaluate various well-known I/O tuning techniques on a new ODMS called Proactive Data Containers (PDC). Our experiments using real science applications and I/O kernels demonstrate that the benefits of these tuning methods with up to 9X I/O performance speedup over the previous version of PDC, and 47X over a highly optimized HDF5 implementation.
Year
DOI
Venue
2019
10.1109/HiPC.2019.00023
2019 IEEE 26th International Conference on High Performance Computing, Data, and Analytics (HiPC)
Keywords
Field
DocType
Object centric,Data Management,Scientific Applications
Hierarchical Data Format,Memory hierarchy,Use case,Computer science,Software,Component-based software engineering,Data management,Distributed computing,Scalability,Speedup
Conference
ISSN
ISBN
Citations 
1094-7256
978-1-7281-4536-5
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Houjun Tang15315.97
Surendra Byna255139.65
Stephen Bailey300.34
Zarija Lukic4272.74
Jialin Liu541.46
Quincey Koziol614114.41
Bin Dong710813.56