We propose a novel framework for multimodal video indexing and retrieval using shrinkage optimized directed information assessment (SODA) as similarity measure. The directed information (DI) is a variant of the classical mutual information which attempts to capture the direction of information flow that videos naturally possess. It is applied directly to the empirical probability distributions of both audio-visual features over successive frames. We utilize RASTA-PLP features for audio feature representation and SIFT features for visual feature representation. We compute the joint probability density functions of audio and visual features in order to fuse features from different modalities. With SODA, we further estimate the DI in a manner that is suitable for high dimensional features $p$ and small sample size $n$ (large $p$ small $n$) between pairs of video-audio modalities. We demonstrate the superiority of the SODA approach in video indexing, retrieval, and activity recognition as compared to the state-of-the-art methods such as hidden Markov models (HMM), support vector machine (SVM), cross-media indexing space (CMIS), and other noncausal divergence measures such as mutual information (MI). We also demonstrate the success of SODA in audio and video localization and indexing/retrieval of data with missaligned modalities.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE : Boosting Structure Consistency for Multispectral and Multimodal Image Registration ABSTRACT: In computer vision and computational photography, multispectral imaging is essential. It is vital to align spectral band
PROJECT TITLE : Multimodal Change Detection in Remote Sensing Images Using an Unsupervised Pixel Pairwise-Based Markov Random Field Model ABSTRACT: The multimodal change detection (CD) problem in remote sensing imaging is addressed
PROJECT TITLE : A New Multi-Atlas Registration Framework for Multimodal Pathological Images Using Conventional Monomodal Normal Atlases ABSTRACT: For tasks like ROI segmentation, anatomical landmark recognition, and so on, information
PROJECT TITLE :Semantic Neighbor Graph Hashing for Multimodal Retrieval - 2018ABSTRACT:Hashing strategies are widely used for approximate nearest neighbor search in recent years due to its computational and storage effectiveness.
PROJECT TITLE :Local Multimodal Serial Analysis for Fusing EEG-fMRI: A New Method to Study Familial Cortical Myoclonic Tremor and EpilepsyABSTRACT:Integrating information of neuroimaging multimodalities, like electroencephalography

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry