The Taub Faculty of Computer Science Events and Talks

The visual object category reports of artificial neural networks (ANNs) are notoriously sensitive to tiny, adversarial image perturbations. Because human category reports (aka human percepts) are thought to be insensitive to those same small-norm perturbations — and locally stable in general — this argues that ANNs are incomplete scientific models of human visual perception. Consistent with this, we show that when small-norm image perturbations are generated by standard ANN models, human object category percepts are indeed highly stable. However, in this very same "human-presumed-stable" regime, we find that robustified ANNs reliably discover low-norm image perturbations that strongly disrupt human percepts. These previously undetectable human perceptual disruptions are massive in amplitude, approaching the same level of sensitivity seen in robustified ANNs. Further, we show that robustified ANNs support precise perceptual state interventions: they guide the construction of low-norm image perturbations that strongly alter human category percepts toward specific prescribed percepts. These observations suggest that for arbitrary starting points in image space, there exists a set of nearby "wormholes", each leading the subject from their current category perceptual state into a semantically very different state. Moreover, contemporary ANN models of biological visual processing are now accurate enough to consistently guide us to those portals.

Short Bio:

Guy is a Computer Vision postdoctoral researcher at the DiCarlo Lab at MIT, interested in the intersection between machine and human vision. His PhD focused on decoding visual experience from brain activity. His current focus is on harnessing contemporary models of primate visual cognition for neural and behavioral modulation. Guy holds a PhD in Computer Science and an MSc in Physics from The Weizmann Institute of Science, and a BSc in Physics-EECS from The Hebrew University of Jerusalem.