PROJECT TITLE :

Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network

ABSTRACT:

The reconstruction of the light field (LF) is primarily hindered by two obstacles: a large disparity and the effect of not following the Lambertian distribution. Traditional methods either address the problem of large disparity by using depth estimation followed by view synthesis or they avoid explicit depth information in order to enable non-Lambertian rendering. However, these methods rarely solve both problems at once within a unified framework. In this paper, we revisit the traditional LF rendering framework and incorporate contemporary Deep Learning techniques into it in order to address both of the aforementioned challenges. To begin, we use analysis to demonstrate that the aliasing problem is the primary factor contributing to the large disparity as well as the non-Lambertian difficulties. Traditional methods of low-frequency rendering typically attempt to reduce aliasing by employing a reconstruction filter in the Fourier domain. However, it is impossible to successfully implement such a filter inside of a deep learning pipeline. Instead, we present a different framework to perform anti-aliasing reconstruction in the image domain, and we analytically demonstrate that this new framework is just as effective in combating the aliasing problem. After that, we embedded the anti-aliasing framework into a deep neural network by designing an integrated architecture and trainable parameters. This allowed us to fully explore the potential of the system. A peculiar training set that consists of both regular and unstructured LFs is utilized during the process of end-to-end optimization that is used to train the network. In comparison to other methods that are considered to be state-of-the-art, the deep learning pipeline that was proposed demonstrates a significant advantage in terms of its ability to solve problems involving large disparities and non-Lambertian variables. In addition to the view interpolation for an LF, we also show that the proposed pipeline is beneficial for the extrapolation of the view of a light field.