Exploiting Efficient and Scalable Shuffle Transfers in Future Data Center Networks - 2015


Distributed computing systems like MapReduce in data centers transfer massive amount of data across successive processing stages. Such shuffle transfers contribute most of the network traffic and make the network bandwidth become a bottleneck. In many commonly used workloads, data flows in such a transfer are highly correlated and aggregated at the receiver side. To lower down the network traffic and efficiently use the available network bandwidth, we propose to push the aggregation computation into the network and parallelize the shuffle and reduce phases. In this paper, we first examine the gain and feasibility of the in-network aggregation with BCube, a novel server-centric networking structure for future data centers. To exploit such a gain, we model the in-network aggregation problem that is NP-hard in BCube. We propose two approximate methods for building the efficient IRS-based incast aggregation tree and SRS-based shuffle aggregation subgraph, solely based on the labels of their members and the data center topology. We further design scalable forwarding schemes based on Bloom filters to implement in-network aggregation over massive concurrent shuffle transfers. Based on a prototype and large-scale simulations, we demonstrate that our approaches can significantly decrease the amount of network traffic and save the data center resources. Our approaches for BCube can be adapted to other servercentric network structures for future data centers after minimal modifications.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :Exploiting Non-Causal CPU-State Information for Energy-Efficient Mobile Cooperative Computing - 2018ABSTRACT:Scavenging the idling computation resources at the large variety of mobile devices, ranging from tiny
PROJECT TITLE :Estimation of Broadband Multiuser Millimeter Wave Massive MIMO-OFDM Channels by Exploiting Their Sparse Structure - 2018ABSTRACT:In millimeter wave (mm-wave) huge multiple-input multiple-output (MIMO) systems, acquiring
PROJECT TITLE :MPiLoc: Self-Calibrating Multi-Floor Indoor Localization Exploiting Participatory Sensing - 2018ABSTRACT:Whereas location is one of the most important context info in mobile and pervasive computing, giant-scale
PROJECT TITLE :Automatic Identification of Driver’s Smartphone Exploiting Common Vehicle-Riding Actions - 2018ABSTRACT:Texting or browsing the net on a smartphone while driving, referred to as distracted driving, considerably
PROJECT TITLE :Exploiting Transistor-Level Reconfiguration to Optimize Combinational circuits - 2017ABSTRACT:Silicon nanowire reconfigurable field impact transistors (SiNW RFETs) abolish the physical separation of n-sort and p-type

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry