PROJECT TITLE :

Cost-Aware Big Data Processing across Geo-distributed Datacenters - 2017

ABSTRACT:

With the globalization of service, organizations continuously turn out giant volumes of information that require to be analysed over geo-dispersed locations. Traditionally central approach that moving all information to one cluster is inefficient or infeasible thanks to the limitations such as the scarcity of wide-space bandwidth and the low latency requirement of information processing. Processing massive knowledge across geo-distributed datacenters continues to gain popularity in recent years. But, managing distributed MapReduce computations across geo-distributed datacenters poses a number of technical challenges: how to allocate information among a choice of geo-distributed datacenters to reduce the communication value, how to determine the Virtual Machine (VM) provisioning strategy that provides high performance and low value, and what criteria ought to be used to pick a datacenter as the final reducer for large information analytics jobs. In this paper, these challenges is addressed by balancing bandwidth price, storage cost, computing price, migration price, and latency cost, between the 2 MapReduce phases across datacenters. We formulate this complex value optimization problem for knowledge movement, resource provisioning and reducer selection into a joint stochastic integer nonlinear optimization drawback by minimizing the five price factors simultaneously. The Lyapunov framework is integrated into our study and an efficient online algorithm that's in a position to attenuate the long-term time-averaged operation price is further designed. Theoretical analysis shows that our online algorithm can offer a close to optimum resolution with a provable gap and will guarantee that the data processing can be completed inside pre-defined bounded delays. Experiments on WorldCup98 internet website trace validate the theoretical analysis results and demonstrate that our approach is shut to the offline-optimum performance and superior to some representative approaches.