Kernel Cross-Modal Factor Analysis for Information Fusion With Application to Bimodal Emotion Recognition


In this paper, we investigate kernel based methods for multimodal information analysis and fusion. We introduce a novel approach, kernel cross-modal factor analysis, which identifies the optimal transformations that are capable of representing the coupled patterns between two different subsets of features by minimizing the Frobenius norm in the transformed domain. The kernel trick is utilized for modeling the nonlinear relationship between two multidimensional variables. We examine and compare with kernel canonical correlation analysis which finds projection directions that maximize the correlation between two modalities, and kernel matrix fusion which integrates the kernel matrices of respective modalities through algebraic operations. The performance of the introduced method is evaluated on an audiovisual based bimodal emotion recognition problem. We first perform feature extraction from the audio and visual channels respectively. The presented approaches are then utilized to analyze the cross-modal relationship between audio and visual features. A hidden Markov model is subsequently applied for characterizing the statistical dependence across successive time segments, and identifying the inherent temporal structure of the features in the transformed domain. The effectiveness of the proposed solution is demonstrated through extensive experimentation.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :A Variational Pansharpening Approach Based on Reproducible Kernel Hilbert Space and Heaviside Function - 2018ABSTRACT:Pansharpening is a vital application in remote sensing image processing. It will increase the
PROJECT TITLE :Cost-Optimal Caching for D2D Networks With User Mobility: Modeling, Analysis, and Computational Approaches - 2018ABSTRACT:Caching well-liked files at the user equipments (UEs) provides an efficient way to alleviate
PROJECT TITLE :Design, Analysis, and Implementation of ARPKI: An Attack-Resilient Public-Key Infrastructure - 2018ABSTRACT:This Transport Layer Security (TLS) Public-Key Infrastructure (PKI) is based on a weakest-link security
PROJECT TITLE :Minority Oversampling in Kernel Adaptive Subspaces for Class Imbalanced Datasets - 2018ABSTRACT:The class imbalance drawback in machine learning occurs when sure classes are underrepresented relative to the others,
PROJECT TITLE : Online Kernel Slow Feature Analysis for Temporal Video Segmentation and Tracking - 2015 ABSTRACT: Slow feature analysis (SFA) is a dimensionality reduction technique that has been linked to how visual brain

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry