Back to overview

Optimizing Capacity Allocation for Big Data Applications in Cloud Datacenters

Type of publication Peer-reviewed
Publikationsform Proceedings (peer-reviewed)
Author Spicuglia Sebastiano, Chen Lydia Y., Birke Robert, Binder Walter,
Project LoadOpt - Workload Characterization and Optimization for Multicore Systems
Show all

Proceedings (peer-reviewed)

Title of proceedings 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM-2015)
Place Ottawa, ON, Canada
DOI 10.1109/inm.2015.7140330


To operate systems cost-effectively, cloud providers not only multiplex applications on the shared infrastructure but also dynamically allocate available resources, such as power and cores. Data intensive applications based on the MapReduce paradigm rapidly grow in popularity and importance in the Cloud. Such big data applications typically have high fan-out of components and workload dynamics. It is no mean feat to deploy and further optimize application performance within (stringent) resource budgets. In this paper, we develop a novel solution, OptiCA, that eases the deployment of big data applications on cloud and the control of application components so that desired performance metrics can be best achieved for any given resource budgets, in terms of core capacities. The control algorithm of OptiCA distributes the available core budget across co-executed applications and components, based on their “effective” demands obtained through non-intrusive profiling. Our proposed solution is able to achieve robust performance, i.e., with very minor degradation, in cases where resource budget decreases rapidly.