The Taub Faculty of Computer Science Events and Talks

At the CVLAB, EPFL, We have developed a multi-people tracking system. Our system is able to reliably track multiple people in a multi-camera setting. The obtained trajectories can be used for understanding individuals and group behavior. For example, we are currently involved in a project whose goal is to understand the behavior of basketball teams and players from multiple video cameras.

Our system is composed of three core components: detection, identification and tracking. The detection component represents the ground floor of the scene as a grid of cells and uses a generative model to estimate the probability of each grid cell to be occupied by a person. This produces what we call a Probability Occupancy Map (POM). Our detection algorithm can effectively handle occlusions and produces POM for each time frame independently, which is then used by the next components. The identification component recognizes the identity of the person according to his color histogram and, in the case of sport matches, his jersey number. In the final tracking component, the multi-people tracking problem is formulated as a multi-commodity network flow problem. The tracker links the detections of people in individual frames across time, while taking into account the appearance constraints. Finally, our tracking system estimates, at each time step, which grid cells are occupied and by whom.

The full system demonstrates excellent results on long and challenging video sequences, including a pedestrian benchmark dataset and several sports datasets. We show that our system works reliably in spite of significant occlusions and delivers metrically accurate trajectories for each tracked individual.

In the talk, I will present in detail the detection and tracking algorithms which we made available for academic use. You are encouraged to try them for your own tracking problems.