Test of Time Paper Award of SIGIR 2014

Professor William Cohen

Cheng Zhai, William Cohen, and John Lafferty won a SIGIR Test of Time Award for their SIGIR 2003 paper "Beyond Independent Relevance: Methods and Evaluation Metrics for Subtopic
Retrieval".  At the time the paper was written, Cheng was a recent LTI graduate, and William and John were faculty in the Language Technologies Institute and the Center for Automated Learning and Discovery (now the Machine Learning Department). 

The paper introduced the idea that a search topic may be composed of many related subtopics, thus a good search engine should select documents that cover as many of the subtopics as possible.  It presented new document ranking algorithms, and new metrics for measuring the effectiveness of a ranking algorithm at covering a set of subtopics.  Although the algorithms and metrics were superseded by newer algorithms and metrics, the idea that a good ranking should cover the latent subtopics as well as the broader search topic became an established element of search engine design and information retrieval research. 

This is the first year that the SIGIR conference has awarded Test of Time Awards.  In this first year, it considered papers from the years 2002 to 2004, selecting Zhai, et al., as the winner.  Three other papers received Honorable Mention.  All four papers will be reprinted in a forthcoming issue of SIGIR Forum.