Bridging the Semantic Gap via Functional Brain Imaging


The multimedia content analysis community has made significant efforts to bridge the gaps between low-level features and high-level semantics perceived by humans. Recent advances in brain imaging and neuroscience in exploring the human brain's responses during multimedia comprehension demonstrated the possibility of leveraging cognitive neuroscience knowledge to bridge the semantic gaps. This paper presents our initial effort in this direction by using functional magnetic resonance imaging (fMRI). Specifically, task-based fMRI (T-fMRI) was performed to accurately localize the brain regions involved in video comprehension. Then, natural stimulus fMRI (N-fMRI) data were acquired when subjects watched the multimedia clips selected from the TRECVID datasets. The responses in the localized brain regions were measured and used to extract high-level features as the representation of the brain's comprehension of semantics in the videos. A novel computational framework was developed to learn the most relevant low-level feature sets that best correlate the fMRI-derived semantic features based on the training videos with fMRI scans, and then the learned model was applied to larger scale TRECVID video datasets without fMRI scans for category classification. Our experimental results demonstrate: 1) there are meaningful couplings between brain's fMRI-derived responses and video stimuli, suggesting the validity of linking semantics and low-level features via fMRI and 2) the computationally learned low-level features can significantly $({rm p}< 0.01)$ improve video classification in comparison with original low-level features and extracted low-level features resulted from well-known feature projection algorithms.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :Bridging the Gap: Using Technology to Capture the Old and Encourage the NewABSTRACT:For the first time in our employer?s history, our workforce has four generations working along simultaneously, every with its unique
PROJECT TITLE :Net2plan: an open source network planning tool for bridging the gap between academia and industryABSTRACT:The plethora of network designing results published in high-ranked journals is a good sign of the success
PROJECT TITLE :Bridging Music Using Sound-Effect InsertionABSTRACT:In this text, the authors provide a general overview of audio music concatenation, then deliberate on how to connect not-therefore-coherent music clips in response
PROJECT TITLE :Basic Principles of MPC for Power Converters: Bridging the Gap Between Theory and PracticeABSTRACT:This article presents the essential principles of operation for model predictive management (MPC), a control methodology
PROJECT TITLE :Experimental studies of influence of different electrodes on bridging in contaminated transformer oilABSTRACT:Contaminated transformer oil has been tested underneath non uniform electric fields and the effect of

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry