The Taub Faculty of Computer Science Events and Talks

Object classification and more generally - semantic perception, are an important aspect in situational awareness in autonomous systems. Recent advances in visual information processing have enabled the use of rich semantic information in critical systems, spurring demand for robust, uncertainty-aware semantic perception. The integration of semantic information with noisy spatial (pose, world geometry) information results in mixed - continuous and discrete state belief, often leading to mixture models that are intractable in the general case. As a result, current methods generally ignore state uncertainty when dealing with semantic information, leading to errors where this uncertainty is manifested. Commonly, the problem is simplified even further by assuming spatial independence among measurements and only dealing with most-probable-class measurements, often discarding richer semantic information and making these methods more prone to noise. This seminar presents an approach for incorporating semantic (object class) information in Bayesian state estimation for robust visual classification of a scene object by a mobile robot operating in a previously unknown, partially observable environment, overcoming limitations of current methods. We make use of rich semantic measurements provided by a Bayesian Neural Network classifier with a measure of uncertainty. Fusion of classifier outputs takes into account viewpoint dependency and spatial correlation among observations, as well as pose uncertainty when these observations are taken. Our experiments confirm an improvement in robustness over state-of-the-art.