Skip to content (access key 's')
Logo of Technion
Logo of CS Department
Logo of CS4People

The Taub Faculty of Computer Science Events and Talks

Machine Learning for Programming Language Processing
event speaker icon
Uri Alon (Ph.D. Thesis Seminar)
event date icon
Tuesday, 08.06.2021, 10:00
event location icon
Zoom Lecture: 3739170675
event speaker icon
Advisor: Prof. Eran Yahav
This talk will focus on structural representations and neural models of source code. I will present a language-agnostic approach for structural language modeling (SLM) of code. This general approach obtains state-of-the-art results in a variety of tasks including code summarization, code captioning, code completion, name prediction, and edit completion, outperforming sequence models (such as textual Transformers and LSTMs) and models based on graph neural networks (GNNs). Studying the reason why GNNs do poorly compared to SLM exposed a fundamental bottleneck that results in a phenomenon that we call "over-squashing". This bottleneck of GNNs provides a new perspective on a phenomenon that was observed for years and still affects existing GNN-based models. I will explain the GNN bottleneck problem and demonstrate its practical implications. The talk summarizes my PhD dissertation, which has been presented in both machine learning and programming language conferences.