Paper Info
Title
Performance Evaluation of an Algorithm-based Asynchronous Checkpoint-Restart Fault Tolerant Application Using Mixed MPI/GPI-2.
Abstract
One of the hardest challenges of the current Big Data landscape is the lack of ability to process huge volumes of information in an acceptable time. The goal of this work, is to ascertain if it is useful to use typical Big Data tools to solve High Performance Computing problems, by exploring and comparing a distributed computing framework implemented on a commodity cluster architecture: the experiment will depend on the computational time required using tools such as Apache Spark. This will be compared to equivalent more traditional approaches such as using a distributed memory model with MPI on a distributed file system such as HDFS (Hadoop Distributed File System) and native C libraries that create an interface to encapsulate this file system functionalities, and using the GPI-2 implementation for the GASPI protocol and itu0027s in-memory checkpointing library to provide an application with Fault Tolerance features. To be more precise, weu0027ve chosen the K-means algorithm as experiment, that will be ran on variable size datasets, and then we will compare the computational run time and time resilience of both approaches.
Year
Venue
Field
2018
arXiv: Distributed, Parallel, and Cluster Computing
Distributed File System,Asynchronous communication,File system,Spark (mathematics),Supercomputer,Computer science,Distributed memory,Algorithm,Fault tolerance,Big data,Distributed computing
DocType
Volume
Citations 
Journal
abs/1804.11312
0
PageRank 
References 
Authors
0.34
0
2
Name
Order
Citations
PageRank
Adrian Bazaga101.01
Michal Pitonák2326.00