PROJECT TITLE :

AdaPool: A Diurnal-Adaptive Fleet Management Framework Using Model-Free Deep Reinforcement Learning and Change Point Detection

ABSTRACT:

In this paper, an adaptive model-free deep reinforcement approach is presented. This approach is able to recognize and adapt to the diurnal patterns that occur in an environment that involves ride-sharing and car-pooling. Because it is agnostic to the timescale of changes in the distribution of experiences, Deep Reinforcement Learning (RL) is prone to catastrophic forgetting. This only holds true in the presence of static environments, despite the fact that RL algorithms are guaranteed to converge to optimal policies in Markov decision processes (MDPs). Nevertheless, the implications of this assumption are quite limited. In many real-world problems, such as ride-sharing, traffic control, and other similar issues, we are dealing with highly dynamic environments, which means that RL methods can only produce decisions that are less than optimal. In order to alleviate this issue in highly dynamic environments, we have taken the following steps: (1) developed a Deep Q Network (DQN) agent that is capable of recognizing diurnal patterns and making informed dispatching decisions according to the changes in the underlying environment; and (2) adopted an online Dirichlet change point detection (ODCP) algorithm to detect the changes in the distribution of experiences. Both of these steps are described in more detail below. The proposed method, as opposed to manually adjusting patterns based on the day of the week, makes an automatic discovery that the MDP has been modified and applies the outcomes of the updated model. In addition to the adaptation logic that is used in dispatching, this paper also proposes a dynamic, demand-aware vehicle-passenger matching and route planning framework. This framework dynamically generates optimal routes for each vehicle based on online demand, vehicle capacities, and locations. In other words, the paper goes beyond just the adaptation logic that is used in dispatching. The evaluation of our strategy using the publicly available data from the New York City Taxi Department shows that it is effective in increasing fleet utilization. Specifically, our strategy allows for less than half of the fleet to be utilized to meet the demand of up to 90 percent of the requests, all while maximizing profits and minimizing idle times.