The Taub Faculty of Computer Science Events and Talks

From minute surface vibrations to very fast-occurring events, the world is rich with phenomena humans cannot perceive. Likewise, most computer vision systems are primarily based on 'conventional' cameras, which were designed to mimic the imaging principle of the human eye, and therefore are equally blind to these ubiquitous phenomena. In this talk, I will show that we can capture these hidden phenomena by creatively building novel vision systems composed of common off-the-shelf components (i.e., cameras and optics) coupled with cutting-edge algorithms. Specifically, I will cover three projects using computational imaging to sense hidden phenomena. First, I will describe the ACam - a camera designed to capture the minute flicker of electric lights ubiquitous in our modern environments. I will show that bulb flicker is a powerful visual cue that enables various applications like scene light source unmixing, reflection separation, and remote analyses of the electric grid itself. Second, I will describe Diffraction Line Imaging, a novel imaging principle that exploits diffractive optics to capture sparse 2D scenes with 1D (line) sensors. The method's applications include capturing fast motions (e.g., actors and particles within a fast-flowing liquid) and structured light 3D scanning with line illumination and line sensing. Lastly, I will present a new approach for sensing minute high-frequency surface vibrations (up to 63kHz) for multiple scene sources simultaneously, using "slow" sensors rated for only 130Hz. Applications include capturing vibration caused by audio sources (e.g., speakers, human voice, and musical instruments) and localizing vibration sources (e.g., the position of a knock on the door). Bio: Mark Sheinin is a Post-doctoral Research Associate at Carnegie Mellon University's Robotic Institute at the Illumination and Imaging Laboratory. He received his Ph.D. in Electrical Engineering from the Technion - Israel Institue of Technology in 2019. His work has received the Best Student Paper Award at CVPR 2017 and the Best Paper Honorable Mention Award at CVPR 2022. He received the Porat Award for Outstanding Graduate Students, the Jacobs-Qualcomm Fellowship in 2017, and the Jacobs Distinguished Publication Award in 2018. His research interests include computational photography and computer vision.