The Taub Faculty of Computer Science Events and Talks

We tackle the problem of multiple image alignment and 3D reconstruction under extreme noise. Photographs acquired in scenes with large intensity variations or in scattering media have significant spatial variance in signal-to-noise-ratio (SNR), where in some image regions it is significantly below the level current methods cope with. Modern alignment schemes, based on similarity measures, feature matching and optical flow are often pairwise, or assume global alignment. Nevertheless, under extreme noise, the alignment success sharply deteriorates, since each image does not contain enough information. Yet, when multiple images are well aligned, the signal emerges from the stack. We aim to restore complex 3D scenes, where global motion cannot be assumed. As the problems of alignment and 3D reconstruction are coupled, we constrain the solution by taking into account only alignments that are geometrically feasible and solve for the entire stack simultaneously. The solution is formulated as a variational problem where only a single variable per pixel, the scene's distance, is optimized. Thus, the complexity of the algorithm is independent of the number of images. Our method outperforms state-of-the-art techniques, as indicated by our simulations and by experiments on real-world scenes.

*MSc seminar under supervision of Dr. Tali Treibitz