Pixel Club: InterpoNet, A Brain Inspired Neural Network for Optical Flow Dense Interpolation

Shay Zweig (Bar-Ilan & Tel-Aviv University)
Tuesday, 27.6.2017, 11:30
EE Meyer Building 1061

Sparse-to-dense interpolation for optical flow is a fundamental phase in the pipeline of most of the leading optical flow estimation algorithms. The current state-of-the-art method for interpolation, EpicFlow, is a local average method based on an edge aware geodesic distance. We propose a new data-driven sparse-to-dense interpolation algorithm based on a fully convolutional network. We draw inspiration from the filling-in process in the visual cortex and introduce lateral dependencies between neurons and multi-layer supervision into our learning process. We also show the importance of the image contour to the learning process. Our method is robust and outperforms EpicFlow on competitive optical flow benchmarks with several underlying matching algorithms. This leads to state-of-the-art performance on the Sintel and KITTI 2012 benchmarks.

Did my BSc In CS in the college of management. My Msc and PHD in neuroscience in the gonda center for brain research in Bar Ilan University. I spent the first half of my PhD in Proffesor Hamutal Slovin's lab investigating the visual system in the primate brain using voltage sensitive dye imaging. The second half was dedicated to deep learning and computer vision under the supervision of Professor Lior Wolf of TAU. Currently I lead the computer vision and ML field in Intuition Robotics, a company that develops a social robot designed to reduce loneliness among the elderly.

Back to the index of events