aHDFS: An Erasure-Coded Data Archival System for Hadoop Clusters - 2017


In this paper, we have a tendency to propose an erasure-coded data archival system known as aHDFS for Hadoop clusters, where RS(k + r; k) codes are utilized to archive data replicas in the Hadoop distributed file system or HDFS. We have a tendency to develop two archival strategies (i.e., aHDFS-Grouping and aHDFS-Pipeline) in aHDFSto speed up the information archival process. aHDFS-Groupinga MapReduce-primarily based data archiving scheme - keeps every mapper's intermediate output Key-Value pairs in a very native key-value store. With the native store in place, aHDFS-Grouping merges all the intermediate key-worth pairs with the identical key into one single key-price try, followed by shuffling the single Key-Value pair to reducers to get final parity blocks. aHDFS-Pipeline forms a knowledge archival pipeline using multiple information node in a Hadoop cluster. aHDFS-Pipeline delivers the merged single key-worth combine to a subsequent node's local key-price store. Last node within the pipeline is accountable for outputting parity blocks. We tend to implement aHDFS in a very real-world Hadoop cluster. The experimental results show that aHDFS-Grouping and aHDFS-Pipeline speed up Baseline's shuffle and cut back phases by a issue of ten and five, respectively. When block size is larger than 32 MB, aHDFS improves the performance of HDFS-RAID and HDFS-EC by approximately 31.eight and p.c, respectively.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE : Secure and Efficient Data Transmission for Cluster-Based Wireless Sensor Networks - 2014 ABSTRACT: Secure data transmission is a critical issue for wireless sensor networks (WSNs). Clustering is an effective
PROJECT TITLE : R3E Reliable Reactive Routing Enhancement for Wireless Sensor Networks - 2014 ABSTRACT: Providing reliable and efficient communication under fading channels is one of the major technical challenges in wireless
PROJECT TITLE : PSR A Lightweight Proactive Source Routing Protocol For Mobile Ad Hoc Networks - 2014 ABSTRACT: Opportunistic data forwarding has drawn much attention in the research community of multihop wireless networking,
PROJECT TITLE : Multi-Core Embedded Wireless Sensor Networks Architecture and Applications - 2014 ABSTRACT: Technological advancements in the silicon industry, as predicted by Moore's law, have enabled integration of billions
PROJECT TITLE : Fast and Secure Multihop Broadcast Solutions for Intervehicular Communication - 2014 ABSTRACT: Intervehicular communication (IVC) is an important emerging research area that is expected to considerably contribute

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry