PROJECT TITLE :
Heterogeneous Environment Aware Streaming Graph Partitioning
With the increasing availability of graph data and widely adopted cloud computing paradigm, graph partitioning has become an economical pre-processing technique to balance the computing workload and cope with the massive scale of input information. Since the cost of partitioning the entire graph is strictly prohibitive, there are some recent tentative works towards streaming graph partitioning that run faster, are simply parallelized, and will be incrementally updated. Most of the prevailing works on streaming partitioning assume that worker nodes at intervals a cluster are homogeneous in nature. Unfortunately, this assumption will not invariably hold. Experiments show that these homogeneous algorithms suffer a significant performance degradation when running at heterogeneous setting. During this paper, we tend to propose a novel adaptive streaming graph partitioning approach to cope with heterogeneous surroundings. We tend to first formally model the heterogeneous computing environment with the thought of the unbalance of computing ability (e.g., the CPU frequency) and communication ability (e.g., the network bandwidth) for each node. Primarily based on this model, we tend to propose a new graph partitioning objective function that aims to minimize the full execution time of the graph-processing job. We tend to then explore some straightforward nevertheless effective streaming algorithms for this objective operate that can achieve balanced and efficient partitioning result. Extensive experiments are conducted on a moderate sized computing cluster with real-world internet and social network graphs. The results demonstrate that the proposed approach achieves significant improvement compared with the state-of-the-art solutions.
Did you like this research project?
To get this research project Guidelines, Training and Code... Click Here