The Taub Faculty of Computer Science Events and Talks

In an Escape-room setting, we study different human behaviors when completing time-constrained tasks involving sequential decision-making and actions. We aim to construct a human mental model linking attention, episodic memory, and hand-object interaction. We record from two egocentric cameras: a head-mounted camera and Gaze-tracking glasses. We also record from up to four third-person cameras. Additionally, we created a detailed 3D map of the room. In this talk, I will discuss the progress we have made and the challenges we faced. Regarding computer vision research, we ask: 1) Can visual synthesis benefit Object Recognition/Affordance prediction? 2) Can we build a superhuman model from observations and simulation? Bio: Jianbo studied Computer Science and Mathematics as an undergraduate at Cornell University where he received his B.A. in 1994. He received his Ph.D. degree in Computer Science from University of California at Berkeley in 1998, for his thesis on Normalize Cuts image segmentation algorithm. He joined The Robotics Institute at Carnegie Mellon University in 1999 as a research faculty. Since 2003, he has been with the Department of Computer & Information Science at the University of Pennsylvania. Jianbo's group is developing vision algorithms for both human and image recognition. Their ultimate goal is to develop computation algorithms to understand human behavior and interaction with objects in video, and to do so at multiple levels of abstractions: from the basic body limb tracking, to human identification, gesture recognition, and activity inference. Jianbo and his group are working to develop a visual thinking model that allows computers to understand their surroundings and achieve higher-level cognitive abilities such as machine memory and learning.