Bridging the Semantic Gap via Functional Brain Imaging


The multimedia content analysis community has made significant efforts to bridge the gaps between low-level features and high-level semantics perceived by humans. Recent advances in brain imaging and neuroscience in exploring the human brain's responses during multimedia comprehension demonstrated the possibility of leveraging cognitive neuroscience knowledge to bridge the semantic gaps. This paper presents our initial effort in this direction by using functional magnetic resonance imaging (fMRI). Specifically, task-based fMRI (T-fMRI) was performed to accurately localize the brain regions involved in video comprehension. Then, natural stimulus fMRI (N-fMRI) data were acquired when subjects watched the multimedia clips selected from the TRECVID datasets. The responses in the localized brain regions were measured and used to extract high-level features as the representation of the brain's comprehension of semantics in the videos. A novel computational framework was developed to learn the most relevant low-level feature sets that best correlate the fMRI-derived semantic features based on the training videos with fMRI scans, and then the learned model was applied to larger scale TRECVID video datasets without fMRI scans for category classification. Our experimental results demonstrate: 1) there are meaningful couplings between brain's fMRI-derived responses and video stimuli, suggesting the validity of linking semantics and low-level features via fMRI and 2) the computationally learned low-level features can significantly $({rm p}< 0.01)$ improve video classification in comparison with original low-level features and extracted low-level features resulted from well-known feature projection algorithms.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here

PROJECT TITLE :Net2plan: an open source network planning tool for bridging the gap between academia and industryABSTRACT:The plethora of network designing results published in high-ranked journals is a good sign of the success
PROJECT TITLE :Bridging Music Using Sound-Effect InsertionABSTRACT:In this text, the authors provide a general overview of audio music concatenation, then deliberate on how to connect not-therefore-coherent music clips in response
PROJECT TITLE : Security Analysis of Handover Key Management in 4G LTESAE Networks - 2014 ABSTRACT: The goal of 3GPP Long Term Evolution/System Architecture Evolution (LTE/SAE) is to move mobile cellular wireless technology
PROJECT TITLE : On the Delay Advantage of Coding in Packet Erasure Networks - 2014 ABSTRACT: We consider the delay of network coding compared to routing with retransmissions in packet erasure networks with probabilistic erasures.
ABSTRACT: A major downside of classification learning is the dearth of ground-truth labeled knowledge. It is usually expensive to label new information instances for coaching a model. To solve this problem, domain adaptation in

Ready to Complete Your Academic MTech Project Work In Affordable Price ?

Project Enquiry