An Implementation Framework for Learning-based Data Placement for Low Latency in Data Center Networks

PROJECT TITLE :

A Learning-based Data Placement Framework for Low Latency in Data Center Networks

ABSTRACT:

The provision of data services with a low latency is becoming an increasingly important challenge for data center applications. In contemporary distributed storage systems, accurate data placement helps cut down on the delay associated with data movement, which in turn can make a significant contribution to the reduction of service latency. Existing data placement solutions frequently make the assumption that the prior distribution of data requests was performed, or they discover this information through trace analysis. Nevertheless, the placement of data is a challenging online decision-making problem due to the dynamic conditions of the network and the time-varying patterns of user requests. When it comes to dealing with dynamic systems, the traditional model-based static solutions are not nearly as effective. We develop a reinforcement learning-based framework called DataBot+, which is capable of automatically learning the optimal placement policies. This is done with an overall consideration of data movement and analytical latency. DataBot+ uses neural networks that have been trained using a variation of Q-learning. These neural networks take the measurements of the real-time data flow as their input, and their output is a value function that estimates the latency in the near future. DataBot+ was designed to allow for instantaneous decision making by being decoupled into two asynchronous production and training components. This ensures that the training delay will not introduce any additional overheads to the process of handling the data flows. The effectiveness of our design is demonstrated by the evaluation results which were driven by traces from the real world.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here