PROJECT TITLE :

Deep Coupled ISTA Network for Multi-Modal Image Super-Resolution

ABSTRACT:

In order to obtain a high-resolution (HR) version of a low-resolution (LR) image, multi-modal image super-resolution (MISR) uses another modality's high-resolution image as guidance. In this research, we propose a new deep network architecture for MISR based on a model-based approach. A new joint multi-modal dictionary learning (JMDL) approach is first introduced to model cross-modal dependency. There are four different dictionaries in JMDL, each with its own set of two transform matrices. The JMDL model is then transformed into a deep neural network known as a "deep coupled ISTA network" by using the "iterative shrinkage and thresholding method" (ISTA). We also present a layer-wise optimization algorithm (LOA) to initialise the network parameters prior to executing back-propagation method, because network initialization is critical in deep network training. We tackle the convex optimization problem by modelling the network initialization as a multi-layer dictionary learning problem. In a study, the proposed LOA was found to reduce training loss and improve reconstruction accuracy. As a last step, we compare our method to various current methods for analysing MISR images. A comparison of the numerical findings for numerous multi-modal scenarios shows that our method regularly surpasses the competition in terms of both quantitative and qualitative metrics.