We propose a novel framework for multimodal video indexing and retrieval using shrinkage optimized directed information assessment (SODA) as similarity measure. The directed information (DI) is a variant of the classical mutual information which attempts to capture the direction of information flow that videos naturally possess. It is applied directly to the empirical probability distributions of both audio-visual features over successive frames. We utilize RASTA-PLP features for audio feature representation and SIFT features for visual feature representation. We compute the joint probability density functions of audio and visual features in order to fuse features from different modalities. With SODA, we further estimate the DI in a manner that is suitable for high dimensional features $p$ and small sample size $n$ (large $p$ small $n$) between pairs of video-audio modalities. We demonstrate the superiority of the SODA approach in video indexing, retrieval, and activity recognition as compared to the state-of-the-art methods such as hidden Markov models (HMM), support vector machine (SVM), cross-media indexing space (CMIS), and other noncausal divergence measures such as mutual information (MI). We also demonstrate the success of SODA in audio and video localization and indexing/retrieval of data with missaligned modalities.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :Semantic Neighbor Graph Hashing for Multimodal Retrieval - 2018ABSTRACT:Hashing strategies are widely used for approximate nearest neighbor search in recent years due to its computational and storage effectiveness.
PROJECT TITLE :Local Multimodal Serial Analysis for Fusing EEG-fMRI: A New Method to Study Familial Cortical Myoclonic Tremor and EpilepsyABSTRACT:Integrating information of neuroimaging multimodalities, like electroencephalography
PROJECT TITLE :Design of a Multimodal EEG-based Hybrid BCI System with Visual Servo ModuleABSTRACT:Current EEG-based brain-computer interface technologies mainly specialize in the way to independently use SSVEP, motor imagery,
PROJECT TITLE :Multimodal Affect Classification at Various Temporal LengthsABSTRACT:Earlier studies have shown that certain emotional characteristics are best observed at completely different analysis-frame lengths. When features
PROJECT TITLE :Multimodal Medical Image Sensor Fusion Framework Using Cascade of Wavelet and Contourlet Transform DomainsABSTRACT:Multimodal medical image fusion is effectuated to minimize the redundancy whereas augmenting the

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry