The Taub Faculty of Computer Science Events and Talks

Henry Taub Distinguished Visitor Lecture: A thorough and convincing evaluation is a core part of every systems paper, and generally involves some form of benchmarking. In my experience as a reviewer of hundreds of papers, there are a number of things people frequently get wrong in their evaluation. I have distilled those into a list of systems benchmarking crimes, most of which are sufficient reason to reject a paper. I will discuss those crimes and how to avoid them.

Bio:
Gernot Heiser is Scientia Professor and John Lions Chair of Operating Systems at the University of New South Wales (UNSW), and leads the Software Systems Research Group at NICTA, Australia's National Centre of Excellence for ICT Research. He joined NICTA at its creation in 2002, and before that was a full-time member of academic staff at UNSW from 1991. His past work included the Mungi single-address-space operating system (OS), several un-broken records in IPC performance, and the best-ever reported performance for user-level device drivers. More recently he led the team at NICTA which produced the world's first formal proof of functional correctness of a protected general-purpose operating-system kernel, the first published sound worst-case execution time analysis of a protected OS kernel, and the synthesis of high-performance device drivers.