Skip to content (access key 's')
Logo of Technion
Logo of CS Department
Logo of CS4People

The Taub Faculty of Computer Science Events and Talks

Pixel Club: Understanding, Improving, And Extending The Contrastive Divergence Method For Training Energy-Based Models
event speaker icon
Omer Yair
event date icon
Tuesday, 17.10.2023, 11:30
event location icon
Room 1061, EE Meyer Building & Zoom Lecture: 92473792641
Recent years have witnessed remarkable advancements in generative models within the realm of computer vision. However, while great progress has been made in implicit generative techniques (e.g. GANs and Diffusion Models), methods that explicitly model the data distribution have been significantly lagging behind. This seminar will present our research on such methods, which are collectively known as Energy-Based Models (EBMs). I will start by revisiting the classical Contrastive Divergence algorithm for training EBMs (Hinton, 2002). The original derivation of this algorithm relied on an unjustified approximation. Here, I will show that this method can be derived in an alternative way, which relies on no approximations, and sheds new light on how and why the CD algorithm works. Based on insights from our analysis, I will then present an improved CD method that substantially narrows the performance gap to the current state-of-the-art techniques. Finally, I will demonstrate how our method can be harnessed for visualizing uncertainties in inverse problems.