PROJECT TITLE :

MetaFlow: A Scalable Metadata Lookup Service for Distributed File Systems in Data Centers - 2018

ABSTRACT:

In massive-scale distributed file systems, efficient metadata operations are vital since most file operations should interact with metadata servers first. In existing distributed hash table (DHT) primarily based metadata management systems, the lookup service could be a performance bottleneck because of its significant CPU overhead. Our investigations showed that the lookup service could cut back system throughput by up to 70 percent, and increase system latency by a issue of up to 8 compared to ideal eventualities. In this Project, we tend to present MetaFlow, a scalable metadata lookup service utilizing software-defined NetWorking (SDN) techniques to distribute lookup workload over network elements. MetaFlow tackles the lookup bottleneck problem by leveraging B-tree, that is constructed over the physical topology, to manage flow tables for SDN-enabled switches. So, metadata requests can be forwarded to acceptable servers using only switches. Extensive performance evaluations in each simulations and testbed showed that MetaFlow increases system throughput by a factor of up to 3.2, and scale back system latency by a factor of up to five compared to DHT-based mostly systems. We conjointly deployed MetaFlow in a distributed file system, and demonstrated vital performance improvement.