אירועים והרצאות בפקולטה למדעי המחשב ע"ש הנרי ומרילין טאוב

Large scale Mega-models are impossible to train using standard hardware, but encompass a vast semantic understanding of our world. In this talk, I explore three ways to leverage the knowledge encompassed in the Recent Large scale"Contrastive Language-Image Pre-training" (CLIP) model, as a foundation to push the boundaries of generative capabilities: - InStyleGAN-NADA, we show how to adapt the StyleGAN generator across a multitudeof domains characterized by diverse styles and shapes. Notably, the native editing capabilities StyleGAN offers are preserved during this adaptation, and can be readily applied to the new domain. - InClipasso, we leverage only the visual (yet abstract and semantic) knowledge encoded in CLIP, to guide an object sketching optimization. The work converts an image of an object to a sketch, allowing varying levels of abstraction while preserving key visual features. - InMotionClip, we introduce a 3D human motion auto-encoder featuring a latent embedding that is disentangled, well behaved, and supports highly semantic textual descriptions, all thanks to CLIP. Specifically, we show how aligning the human motion manifold to CLIP space implicitly infuses the extremely rich semantic knowledge of CLIP into the manifold, even though it has not seen any temporal data during training. Bio: Dr. Amit H. Bermano is a senior lecturer (assistant professor) at the Blavatnik School of Computer Science in Tel-Aviv University since 2018. His research focuses on Computer Graphics, Computer Vision, and Computational Fabrication, using tools from Machine Learning and Geometry Processing. Previously, he was a postdoctoral researcher at the Princeton Graphics Group, and a postdoctoral researcher at Disney Research Zurich. He has conducted his doctoral studies at ETH Zurich, in collaboration with the computational materials group of Disney Research Zurich. His Master’s and Bachelors’s degrees were obtained at the Technion - Israel Institute of Technology