Back to Top

■ A Framework for Efficient Energy Scheduling of Spark Workloads

A Framework for Efficient Energy Scheduling of Spark Workloads, Stathis Maroulis, Nikos Zacheilas, Vana Kalogeraki , IEEE ICDCS 2017, Atlanta, GA, USA, June 5 - 8, 2017 (poster)
Nowadays distributed processing frameworks like Apache Spark have been successfully used for the execution of big data applications. Despite their wide adoption little work has been done in terms of controlling the applications' energy consumption. Datacenters contribute over 2 % of the total US electric usage therefore minimizing the energy utilization of Spark application can be extremely helpful. Solving this energy consumption problem requires the scheduling of Spark applications in an energy-efficient way. However, the problem is challenging as we also have to consider application performance requirements. In this work, we provide the overview of a novel framework that orchestrates the execution order of Spark applications, exploiting DVFS to tune the computing nodes CPU frequencies in order to minimize the energy consumption and satisfy application's performance requirements. Our early experimental results illustrate the working and benefits of our framework.
Bibtex Entry.
  author    = {Stathis Maroulis and
               Nikos Zacheilas and
               Vana Kalogeraki},
  title     = {A Framework for Efficient Energy Scheduling of Spark Workloads},
  booktitle = {37th {IEEE} International Conference on Distributed Computing Systems,
               {ICDCS} 2017, Atlanta, GA, USA, June 5-8, 2017},
  pages     = {2614--2615},
  year      = {2017},
  crossref  = {DBLP:conf/icdcs/2017},
  url       = {},
  doi       = {10.1109/ICDCS.2017.179},
  timestamp = {Fri, 21 Jul 2017 13:46:43 +0200},
  biburl    = {},
  bibsource = {dblp computer science bibliography,}