אירועים והרצאות בפקולטה למדעי המחשב ע"ש הנרי ומרילין טאוב

Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger. For example, it is known that deep networks are exponentially efficient with respect to shallow networks, in the sense that a shallow network must grow exponentially large in order to approximate the functions represented by a deep network of polynomial size. In this work, we extend the study of expressive efficiency to the attribute of network connectivity and in particular to the effect of "overlaps" in the convolutional process, i.e., when the stride of the convolution is smaller than its filter size (receptive field). Our analysis shows that having overlapping local receptive fields, and more broadly denser connectivity, results in an exponential increase in the expressive capacity of neural networks. Moreover, while denser connectivity can increase the expressive capacity, we show that the most common types of modern architectures already exhibit exponential increase in expressivity, without relying on fully-connected layers. Joint work with Or Sharir Short Bio: ========== Amnon Shashua holds the Sachs chair in computer science at the Hebrew University. He received his Ph.D. degree in 1993 from the AI lab at MIT working on computational vision where he pioneered work on multiple view geometry and the recognition of objects under variable lighting. His work on multiple view geometry received best paper awards at the ECCV 2000, the Marr prize in ICCV 2001 and the Landau award in exact sciences in 2005. His work on Graphical Models received a best paper award at the UAI 2008. Prof. Shashua was the head of the School of Engineering and Computer Science at the Hebrew University of Jerusalem during the term 2003-2005. He is also well known on founding startup companies in computer vision and his latest brainchild Mobileye employs today 250 people developing systems-on-chip and computer vision algorithms for detecting pedestrians, vehicles, and traffic signs for driving assistance systems. For his industrial contributions prof. Shashua received the 2004 Kaye Innovation award from the Hebrew University. ============================= Refreshments will be served from 10:15 Lecture starts at 10:30