The Taub Faculty of Computer Science Events and Talks

Single Image Super-Resolution (SISR) aims to recover a high-resolution image from a given low resolution version of it (the given image is assumed to be a blurred, down- sampled and noisy version of the original image). Video Super Resolution (VSR) targets series of given images, aiming to fuse them to create a higher resolution outcome. Although SISR and VSR seem to have a lot in common, as only the input domain changes between the two, most SISR algorithms do not have a simple extension to VSR, apart for the trivial option of applying the SISR for each frame separately. The VSR task is considered to be a more challenging inverse problem, mainly due to its reliance on a sub-pixel accurate motion estimation, which has no parallel in SISR. Another complication is the dynamics of the video, often addressed by simply generating a single frame instead of a complete output sequence. We suggest an appealing alternative to the above that leads to a simple and robust Super-Resolution framework that can be applied to SISR and then easily extended to VSR. Our work relies on the observation that image and video denoising are well-managed and very effectively treated by a variety of methods, many of which not yet effectively adapted to the super-resolution task. We exploit the Plug-and-Play framework and the recently introduced Regularization-by-Denoising (RED) approach that extends it, and show how to use these denoisers in order to handle the SISR and the VSR problems. This way, we benefit from the effectiveness and efficiency of existing image/video denoising algorithms, while solving much more challenging problems. We test our SISR framework against the NCSR algorithm that solves for denoising and super-resolution separately, and show how its denoiser can be used in order to perform highly effective super-resolution. Then we turn to video, harnessing the VBM3D video denoiser, we compare our results to the ones obtained by the DeepSR and 3DSKR algorithms, showing a tendency to a higher-quality output and a much faster processing.