PROJECT TITLE :

Action-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition

ABSTRACT:

However, convolutional neural networks (CNNs) have yet to attain the same spectacular results in video action detection as in image recognition. This is in part due to CNN's failure to simulate long-range temporal structures, particularly those involving specific action phases that are important to human action recognition. Spatiotemporal vector of locally aggregated descriptors (ActionS-ST-VLAD) is proposed in this study to aggregate meaningful deep features over the full video based on adaptive segment feature sampling and action-stage (ActionS) emphasis (AVFS-ASFS). With the use of AVFS-ASFS, keyframe features are selected and deep features are automatically divided into segments with the features in each segment belonging to a temporally coherent ActionS. An advanced flow-guided warping technique is then used to identify and eliminate duplicate feature maps, while a similarity weight is used to aggregate the informative ones. The RGBF modality is used to record motion-sensitive regions in the RGB images that correspond to the activity of the subject. Four public benchmarks - HMDB51, UCF101, Kinetics and ActivityNet - are extensively tested for review. For video-based action detection, results reveal that our method is able to efficiently pool useful deep information spatiotemporally, resulting in the best possible results.