PROJECT TITLE :
Performance Optimization for Managing Massive Numbers of Small Files in Distributed File Systems
The processing of huge numbers of little files is a challenge in the look of distributed file systems. Currently, the combined-block-storage approach is prevalent. However, the approach employs the ancient file systems like ExtFS and could cause inefficiency when accessing tiny files randomly located within the disk. This paper focuses on optimizing the performance of knowledge servers in accessing massive numbers of tiny files. We present a Flat Light-weight File System (iFlatLFS) to manage little files, which relies on a straightforward metadata scheme and a flat storage design. iFlatLFS is designed to substitute the traditional file system on information servers and will be deployed underneath distributed file systems that store large numbers of tiny files. iFlatLFS can greatly simplify the original information access procedure. The new metadata proposed in this paper occupies only a fraction of the metadata size primarily based on ancient file systems. We tend to have implemented iFlatLFS in CentOS five.5 and integrated it into an open source Distributed File System (DFS), called Taobao FileSystem (TFS), which is developed by a top B2C service supplier, Alibaba, in China and is managing over twenty eight.six billion tiny photos. We have conducted extensive experiments to verify the performance of iFlatLFS. The results show that when the file size ranges from 1 to 64 KB, iFlatLFS is quicker than Ext4 by forty eight and 54 p.c on average for random read and write in the DFS setting, respectively. Moreover, once iFlatLFS is integrated into TFS, iFlatLFS-based TFS is faster than the existing Ext4-primarily based TFS by forty five and forty nine p.c on average for random browse access and hybrid access (the combo of browse and write accesses), respectively.
Did you like this research project?
To get this research project Guidelines, Training and Code... Click Here