אלון בריפמן, הרצאה סמינריונית למגיסטר
יום שלישי, 19.6.2018, 11:00
Single Image Super-Resolution (SISR) aims to recover a high-resolution image from a
given low resolution version of it (the given image is assumed to be a blurred, down-
sampled and noisy version of the original image). Video Super Resolution (VSR) targets
series of given images, aiming to fuse them to create a higher resolution outcome.
Although SISR and VSR seem to have a lot in common, as only the input domain
changes between the two, most SISR algorithms do not have a simple extension to
VSR, apart for the trivial option of applying the SISR for each frame separately. The
VSR task is considered to be a more challenging inverse problem, mainly due to its
reliance on a sub-pixel accurate motion estimation, which has no parallel in SISR.
Another complication is the dynamics of the video, often addressed by simply generating
a single frame instead of a complete output sequence.
We suggest an appealing alternative to the above that leads to a simple and robust Super-Resolution
framework that can be applied to SISR and then easily extended to VSR. Our work
relies on the observation that image and video denoising are well-managed and very
effectively treated by a variety of methods, many of which not yet effectively adapted
to the super-resolution task. We exploit the Plug-and-Play framework and the recently
introduced Regularization-by-Denoising (RED) approach that extends it, and show how
to use these denoisers in order to handle the SISR and the VSR problems. This way, we
benefit from the effectiveness and efficiency of existing image/video denoising algorithms,
while solving much more challenging problems. We test our SISR framework against
the NCSR algorithm that solves for denoising and super-resolution separately, and show
how its denoiser can be used in order to perform highly effective super-resolution. Then
we turn to video, harnessing the VBM3D video denoiser, we compare our results to
the ones obtained by the DeepSR and 3DSKR algorithms, showing a tendency to a
higher-quality output and a much faster processing.