Name
Title | ||
|---|---|---|
Finding the Big Data Sweet Spot: Towards Automatically Recommending Configurations for Hadoop Clusters on Docker Containers |
Abstract | ||
|---|---|---|
The complexity of cloud-based analytics environments threatens to undermine their otherwise tremendous values. In particular, configuring such environments presents a great challenge. We propose to alleviate this issue with an engine that recommends configurations for a newly submitted analytics job in an intelligent and timely manner. The engine is rooted in a modified k-nearest neighbor algorithm, which finds desirable configurations from similar past jobs that have performed well. We apply the method to configuring an important class of analytics environments: Hadoop on container-driven clouds. Preliminary evaluation suggests up to 28% performance gain could result from our method. |
Year | DOI | Venue |
|---|---|---|
2015 | 10.1109/IC2E.2015.101 | IC2E |
Keywords | Field | DocType |
big data,engines,cloud computing,linux,resource management | Cluster (physics),Computer science,Analytics,Big data,Database,Cloud computing big data,Cloud computing | Conference |
Citations | PageRank | References |
10 | 0.78 | 13 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Rui Zhang | 1 | 381 | 86.83 |
| Min Li | 2 | 10 | 0.78 |
| Dean Hildebrand | 3 | 146 | 14.96 |