The Taub Faculty of Computer Science Events and Talks

YouTube movies, live streaming, TV broadcast, conference calls and more - there is no doubt that video sequences are abundant and in everyday use. However, the quality of these videos is rarely satisfactory. This may be the result of network limitations, low-quality imaging devices and more.

Improving the quality of videos has been long studied by the research community. Rather than independently improving the quality of each image in the sequence, it has been observed that temporal consideration can greatly improve the outcome, due to details being shown in several consecutive images. In order to take advantage of this observation, existing research till recently focused on estimating the motion between the images. However, motion estimation is quite a complex task, and might result in errors, which degrade the overall quality of the video enhancement process.

In the last couple of years, there has been a trend of applying temporal filtering to videos while bypassing the need for motion estimation. In this talk, we will present two different tasks able to rely on this new approach. We will start by briefly discussing the removal of noise from video, using sparse and redundant representations modeling of images.

Using the intuition gained from this problem, we will turn to one of the holy grails of video processing - super resolution. Super-resolution (SR) deals with the ability to merge a sequence of low-quality images into one (or a sequence of) high-quality image, having better optical resolution and with new details visible. Unfortunately, though existing for over 20 years, SR has suffered greatly from the need for extremely accurate motion estimation. SR has been applicable to only a small portion of the sequences requiring it, as in most sequences, the motion patterns are too complex to be accurately estimated by existing means.

In the lion's share of this talk we will show that it is possible to circumvent the need for accurate deterministic motion estimation by replacing it with crude, probabilistic motion estimation. This allows us to improve the quality of sequences previously considered out of the realm of super-resolution. We show some examples, both synthetic and on actual broadcast data, demonstrating the ability of this approach.

This lecture summarizes portions of a PhD. research by Matan Protter, under the supervision of Prof. Michael Elad.