Deep Coupled ISTA Network for Multi-Modal Image Super-Resolution

PROJECT TITLE :

Deep Coupled ISTA Network for Multi-Modal Image Super-Resolution

ABSTRACT:

In order to obtain a high-resolution (HR) version of a low-resolution (LR) image, multi-modal image super-resolution (MISR) uses another modality's high-resolution image as guidance. In this research, we propose a new deep network architecture for MISR based on a model-based approach. A new joint multi-modal dictionary learning (JMDL) approach is first introduced to model cross-modal dependency. There are four different dictionaries in JMDL, each with its own set of two transform matrices. The JMDL model is then transformed into a deep neural network known as a "deep coupled ISTA network" by using the "iterative shrinkage and thresholding method" (ISTA). We also present a layer-wise optimization algorithm (LOA) to initialise the network parameters prior to executing back-propagation method, because network initialization is critical in deep network training. We tackle the convex optimization problem by modelling the network initialization as a multi-layer dictionary learning problem. In a study, the proposed LOA was found to reduce training loss and improve reconstruction accuracy. As a last step, we compare our method to various current methods for analysing MISR images. A comparison of the numerical findings for numerous multi-modal scenarios shows that our method regularly surpasses the competition in terms of both quantitative and qualitative metrics.

Did you like this research project?

To get this research project Guidelines, Training and Code... Click Here