Performance Optimization for Managing Massive Numbers of Small Files in Distributed File Systems


The processing of huge numbers of little files is a challenge in the look of distributed file systems. Currently, the combined-block-storage approach is prevalent. However, the approach employs the ancient file systems like ExtFS and could cause inefficiency when accessing tiny files randomly located within the disk. This paper focuses on optimizing the performance of knowledge servers in accessing massive numbers of tiny files. We present a Flat Light-weight File System (iFlatLFS) to manage little files, which relies on a straightforward metadata scheme and a flat storage design. iFlatLFS is designed to substitute the traditional file system on information servers and will be deployed underneath distributed file systems that store large numbers of tiny files. iFlatLFS can greatly simplify the original information access procedure. The new metadata proposed in this paper occupies only a fraction of the metadata size primarily based on ancient file systems. We tend to have implemented iFlatLFS in CentOS five.5 and integrated it into an open source Distributed File System (DFS), called Taobao FileSystem (TFS), which is developed by a top B2C service supplier, Alibaba, in China and is managing over twenty eight.six billion tiny photos. We have conducted extensive experiments to verify the performance of iFlatLFS. The results show that when the file size ranges from 1 to 64 KB, iFlatLFS is quicker than Ext4 by forty eight and 54 p.c on average for random read and write in the DFS setting, respectively. Moreover, once iFlatLFS is integrated into TFS, iFlatLFS-based TFS is faster than the existing Ext4-primarily based TFS by forty five and forty nine p.c on average for random browse access and hybrid access (the combo of browse and write accesses), respectively.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :A Machine Learning Approach for Tracking and Predicting Student Performance in Degree Programs - 2018ABSTRACT:Accurately predicting students' future performance based on their ongoing academic records is crucial
PROJECT TITLE :CaL: Extending Data Locality to Consider Concurrency for Performance Optimization - 2018ABSTRACT:Massive information applications demand a higher memory performance. Information Locality has been the main target
PROJECT TITLE :Performance Analysis of Sequential Detection of Primary User Number Based on Multihypothesis Sequential Probability Ratio Test - 2018ABSTRACT:In cognitive radio networks, a priori data on the quantity of primary
PROJECT TITLE :Performance Analysis of a New Calibration Method for Fiber Nonlinearity Compensation - 2018ABSTRACT:Digital signal processing for fiber nonlinearity compensation could be a key enabler for the ever-increasing demand
PROJECT TITLE :Impact of RF–Switch Insertion Loss on the Performance of Space Modulation Techniques - 2018ABSTRACT:The RF switch has been highlighted recently as a core element for hardware implementations of the transmitters

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry