Kernel Cross-Modal Factor Analysis for Information Fusion With Application to Bimodal Emotion Recognition


In this paper, we investigate kernel based methods for multimodal information analysis and fusion. We introduce a novel approach, kernel cross-modal factor analysis, which identifies the optimal transformations that are capable of representing the coupled patterns between two different subsets of features by minimizing the Frobenius norm in the transformed domain. The kernel trick is utilized for modeling the nonlinear relationship between two multidimensional variables. We examine and compare with kernel canonical correlation analysis which finds projection directions that maximize the correlation between two modalities, and kernel matrix fusion which integrates the kernel matrices of respective modalities through algebraic operations. The performance of the introduced method is evaluated on an audiovisual based bimodal emotion recognition problem. We first perform feature extraction from the audio and visual channels respectively. The presented approaches are then utilized to analyze the cross-modal relationship between audio and visual features. A hidden Markov model is subsequently applied for characterizing the statistical dependence across successive time segments, and identifying the inherent temporal structure of the features in the transformed domain. The effectiveness of the proposed solution is demonstrated through extensive experimentation.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE : Scaling Up Generalized Kernel Methods ABSTRACT: Over the course of the past two decades, kernel methods have enjoyed a great deal of success. During this era of big data, the collection of data has experienced
PROJECT TITLE : Robust Variational Learning for Multiclass Kernel Models With Stein Refinement ABSTRACT: The ability of kernel-based models to generalize well is impressive, but the vast majority of them, including the SVM, are
PROJECT TITLE : Efficient Algorithms for Kernel Aggregation Queries ABSTRACT: Kernel functions provide assistance for a wide variety of application types, including those that require activities such as density estimation, classification,
PROJECT TITLE : Composite Kernel of Mutual Learning on Mid-Level Features for Hyperspectral Image Classification ABSTRACT: The effectiveness of the algorithm for machine learning can be enhanced by training multiple models and
PROJECT TITLE : Attention in Reasoning Dataset, Analysis, and Modeling ABSTRACT: Although attention has become an increasingly popular component in deep neural networks for the purpose of both interpreting data and improving

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry