PROJECT TITLE :

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

ABSTRACT:

The long-term goal of image restoration and manipulation is to acquire a solid understanding of image priors. Existing methods, such as deep image prior (DIP), are capable of capturing low-level image statistics; however, there are still gaps on the path to developing an image prior that is capable of capturing rich image semantics, such as color, spatial coherence, textures, and high-level concepts. This research demonstrates an efficient method for utilizing the image prior that is captured by a generative adversarial network (GAN) that has been trained on a large number of natural images. As can be seen in Figure 1, the deep generative prior (DGP) generates compelling results when applied to a variety of degraded images in order to restore missing semantics such as color, patch, and resolution. In addition to that, it enables a wide variety of image manipulations, such as image morphing, random image jittering, and category transfer. The assumption of existing GAN inversion methods, which have a tendency to fix the generator, has been relaxed, which has made it possible to perform such highly flexible restoration and manipulation. Notably, we make it possible for the generator to be fine-tuned on the fly in a progressive manner that is regularized by the feature distance that is obtained by the discriminator in GAN. We show that these straightforward and applicable modifications help preserve the reconstruction to remain in the manifold of nature images, which ultimately leads to a more accurate and faithful reconstruction of real images.