Skip to content (access key 's')
Logo of Technion
Logo of CS Department
Logo of CS4People

The Taub Faculty of Computer Science Events and Talks

Window-Based Distribution Shift Detection for Deep Neural Networks‏
event speaker icon
Guy Bar-Shalom (M.Sc. Thesis Seminar)
event date icon
Wednesday, 13.03.2024, 10:30
event location icon
Zoom Lecture:97086749422 & Meyer 1061
event speaker icon
Advisor: Prof. Ran El-Yaniv and Prof. Haggai Maron
To deploy and operate deep neural models in production, the quality of their predictions, which might be contaminated benignly or manipulated maliciously by input distributional deviations, must be monitored and assessed. Specifically, we study the case of monitoring the healthy operation of a deep neural network (DNN) receiving a stream of data, with the aim of detecting input distributional deviations over which the quality of the network’s predictions is potentially damaged. Using selective prediction principles, we propose a distribution deviation detection method for DNNs. The proposed method is derived from a tight coverage generalization bound computed over a sample of instances drawn from the true underlying distribution. Based on this bound, our detector continuously monitors the operation of the network over a test window and fires off an alarm whenever a deviation is detected. Our novel detection method performs on-par or better than the state-of-the-art, while consuming substantially lower computation time (five orders of magnitude reduction) and space complexity. Unlike previous methods, which require at least linear dependence on the size of the source distribution for each detection, rendering them inapplicable to “Google-Scale” datasets, our approach eliminates this dependence, making it suitable for real-world applications. Code is available at