The Taub Faculty of Computer Science Events and Talks

The driving force behind convolutional networks - the most successful deep learning architecture to date, is their expressive power. Despite its wide acceptance and vast empirical evidence, formal analyses supporting this belief are scarce. The primary notions for formally reasoning about expressiveness are efficiency and inductive bias. Efficiency refers to the ability of a network architecture to realize functions that require an alternative architecture to be much larger. Inductive bias refers to the prioritization of some functions over others given prior knowledge regarding a task at hand. Through an equivalence to hierarchical tensor decompositions, we study the expressive efficiency and inductive bias of various architectural features in convolutional networks (depth, width, pooling geometry etc). Our results shed light on the demonstrated effectiveness of convolutional networks, and in addition, provide new tools for network design.

The talk is based on a series of works published in COLT, ICML, CVPR and ICLR, with collaborators Or Sharir, Ronen Tamari, David Yakira, Yoav Levine and Amnon Shashua.

Bio:
Nadav Cohen is concluding his PhD at the School of Computer Science and Engineering in the Hebrew University of Jerusalem, under the supervision of Prof. Amnon Shashua. His research focuses on the theoretical and algorithmic foundations of deep learning. In particular, he is interested in the application of tensor analysis for the study of convolutional network architectures. Nadav holds a BSc in electrical engineering and a BSc in mathematics (both summa cum laude), conducted under the Technion Excellence Program for distinguished undergraduates. For his research in graduate studies, he received a number of awards, including the Google Doctoral Fellowship in Machine Learning and the Rothschild Postdoctoral Fellowship. Starting September 2017, Nadav will be a postdoctoral member at the Institute for Advanced Study (IAS) in Princeton, hosted by Prof. Sanjeev Arora, Elad Hazan and Yoram Singer.