PROJECT TITLE :

Predicting Appearance of Vehicles From Blind Spots Based on Pedestrian Behaviors at Crossroads

ABSTRACT:

Conventional methods of prediction for traffic scenes focus primarily on determining the future states of visible objects (i.e., those that are not in blind spots) based on their current observations. This study focused on predicting the future states of objects that are located in blind spots, such as those that are outside the field of view or in occluded regions, using the observations that are currently being made of other objects that are visible. A method for predicting the appearance of vehicles from a blind spot based on the behaviors of visible pedestrians who observe vehicles in the blind spot is one that we have proposed. This method takes into account the observations made by those pedestrians. The method that we have proposed makes use of a spatiotemporal 3D Convolutional Neural Network and learns the behaviors of pedestrians in order to make predictions. The method uses Pose Estimation and Semantic Segmentation in order to explicitly represent the subtle motions that are made by pedestrians as well as the environments that they are in. We constructed two datasets of videos that capture real-world traffic scenes so that we could conduct evaluation experiments. Cameras, both with and without ego-motion tracking, are used to collect the datasets. With the help of the datasets, we carried out experiments not only on more straightforward configurations, but also on more realistic traffic environments. The following inferences are possible to make on the basis of the findings from the experiment: I our proposed method achieved a high performance in our prediction task at a level comparable to that of humans. It predicted the appearance of vehicles from blind spots more than 1.5 seconds before they actually appeared in the scene. (ii) Explicit representations of pose and semantic masks captured information that was complementary to RGB videos, and assembling the representations improved the prediction performance. This was accomplished by ensembling the representations. (iii) In order to achieve accurate prediction in the videos captured by driving cars, it is necessary to fine-tune the models using videos that contain ego-motions.