HDM: A Composable Framework for Big Data Processing


Frameworks like MapReduce and Spark have been established in recent years to make constructing Big Data programs and applications easier. The jobs in these frameworks, on the other hand, are only loosely specified and bundled as executable jars, with no functionality exposed or explained.

As a result, deployed jobs are not naturally composable and reusable for future development. It also makes it difficult to add optimizations to the data flow of job sequences and pipelines. The Hierarchically Distributed Data Matrix (HDM) is a functional, strongly-typed data representation for creating composable Big Data applications, which we discuss in this work.

A runtime framework is included with HDM to help with application execution, integration, and management on distributed infrastructures. Multiple optimizations are implemented to increase the performance of running HDM jobs based on the functional data dependency graph of HDM.

When compared to the current state of the art, Apache Spark, the experimental findings demonstrate that our optimizations can reduce Job-Completion-Time by 10 to 40% for various types of applications.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :HDM: A Composable Framework for Big Data Processing - 2018ABSTRACT:Over the past years, frameworks like MapReduce and Spark are introduced to ease the task of developing massive knowledge programs and applications.
PROJECT TITLE : Secure and Efficient Data Transmission for Cluster-Based Wireless Sensor Networks - 2014 ABSTRACT: Secure data transmission is a critical issue for wireless sensor networks (WSNs). Clustering is an effective
PROJECT TITLE : R3E Reliable Reactive Routing Enhancement for Wireless Sensor Networks - 2014 ABSTRACT: Providing reliable and efficient communication under fading channels is one of the major technical challenges in wireless
PROJECT TITLE : PSR A Lightweight Proactive Source Routing Protocol For Mobile Ad Hoc Networks - 2014 ABSTRACT: Opportunistic data forwarding has drawn much attention in the research community of multihop wireless networking,
PROJECT TITLE : Multi-Core Embedded Wireless Sensor Networks Architecture and Applications - 2014 ABSTRACT: Technological advancements in the silicon industry, as predicted by Moore's law, have enabled integration of billions

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry