The Taub Faculty of Computer Science Events and Talks

In learning with privileged information (Vapnik, 2006) the labeled training examples have two views, primary and secondary, and the test examples have only a primary view. The goal of the learner is to leverage the information from these two views in order to build an accurate classifier in the primary view.

SVM+ algorithm, introduced by Vapnik, is a major tool for learning with privileged information. Recently Vapnik, Vashist and Pavlovich (2009) showed that by utilizing the information from the secondary view SVM+ can significantly outperform the widely used SVM algorithm.

We develop two algorithms for solving the optimization problem of SVM+. The first algorithm performs block coordinate descent and is an extension of the SMO algorithm for solving the optimization problem of SVM. The second algorithm performs optimization by using conjugate directions and is significantly faster than the first one.

We also apply the conjugate direction method for solving the optimization problem of SVM. In some regimes (e.g., when SVM is required to generate a classifier with small empirical error) the resulting optimization algorithm is significantly faster than SMO.

Joint work with Leon Bottou and Vladimir Vapnik.