Unsupervised latent variable models serve as highly effective tools for representing complex data such as images or videos, relevant for applications such as robotic manipulation, video generation, novelty detection, and many more. Variational Autoencoders (VAEs) provide compact latent representations with stability and efficiency. In this talk, we will explore modern VAEs that mitigate shortcomings of classical approaches such as blurry images, and can be used as a basis for strong world models. The first paper (CVPR 2021 Oral) introduces “Soft-IntroVAE” , a refined approach to introspective variational autoencoders, enhancing training stability and theoretical insights while showcasing its applications. The second paper (ICML 2022) presents “Deep Latent Particles (DLP)” for unsupervised image representation learning, a new VAE where the latent space is keypoint-based, offering disentangled object features, uncertainty estimation, and versatile applications. Building on DLP, the third paper (TMLR 2024) presents “DDLP”, a novel extension to video prediction, manipulation and generation, where a differentiable tracking module is employed over particles to drive the dynamics modeling. We conclude by highlighting recent applications of these representations in deep reinforcement learning and biomedical domains, demonstrating their broad impact and potential for future research.
Tal (https://taldatech.github.io) is a final-year Ph.D. student in the Electrical and Computer Engineering faculty at the Technion, where he earned his B.Sc. and M.Sc., under the supervision of Prof. Aviv Tamar. He is the winner of The Miriam and Aaron Gutwirth Memorial and The Irwin and Joan Jacobs Ph.D fellowships. His research interests include unsupervised representation learning, generative modeling and reinforcement learning.
Ph.D. Under the supervision of Prof. Aviv Tamar.
