The Taub Faculty of Computer Science Events and Talks

Image and video denoising has been an area of research interest for decades. This talk will present three novel methods that take the denoising field a step forward. All the proposed methods in this work strongly rely on exploiting non-local self-similarity using patch matching. The first method, termed LIDIA [1], has two contributions. First, we propose a low-weight architecture that achieves near state-of-the-art performance. Our architecture relies on patch matching and separable processing. Second, we introduce two simple and highly efficient methods for adapting the network to the input image, for boosting the denoising performance while addressing novel visual content that deviates from the training data. The next method, termed PaCNet [2], is inspired by LIDIA, proposing a novel algorithm for video denoising. As in LIDIA, PaCNet relies on patch matching and separable processing. The proposed method uses the matched patches for constructing patch-craft frames, employing the latter as an augmentation of virtual frames for supporting the denoising task. Our algorithm achieves state-of-the-art performance, surpassing the competitors on average by about 0.7 dB. The third method [3], inspired by both LIDIA and PaCNet, proposes a novel self-supervised training technique suitable for the removal of unknown correlated noise. The proposed approach neither requires knowledge of the noise model nor access to ground-truth targets. We assume that the noise is additive, zero mean, but not necessarily Gaussian, and one that could be short-range spatially correlated. Examples of such noise could be Gaussian correlated noise, shot noise passed through a linear space-invariant system, or real image noise in digital cameras. We demonstrate superior denoising performance compared to leading alternative self-supervised denoising methods. [1] G. Vaksman, M. Elad, and P. Milanfar. Lidia: Lightweight learned image denoising with instance adaptation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2220-2229, 2020. [2] G. Vaksman, M. Elad, and P. Milanfar. Patch craft: Video denoising by deep modeling and patch matching. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2137-2146, 2021. [3] G. Vaksman and M. Elad. Patch-Craft Self-Supervised Training for Correlated Image Denoising. To appear in the Conference on Computer Vision and Pattern Recognition (CVPR), 2023.