Home > Awards, ECIR2009, Information Retrieval Models, Portfolio Theory > Best Paper Prize in ECIR2009

Best Paper Prize in ECIR2009


The Efficient Frontier in document ranking

Jun Wang, Mean-Variance Analysis: A New Document Ranking Theory in Information Retrieval, ECIR2009

This paper concerns document ranking in information retrieval – particularly collaborative filtering and recommender systems . In information retrieval systems, the widely accepted probability ranking principle (PRP) suggests that, for optimal retrieval, documents should be ranked in order of decreasing probability of relevance. In this paper, we present a new document ranking paradigm, arguing that a better, more general solution is to optimize top-n ranked documents as a whole, rather than ranking them independently. Inspired by the Modern Portfolio Theory in finance, we quantify a ranked list of documents on the basis of its expected overall relevance (mean) and its variance; the latter serves as a measure of risk, which was rarely studied for document ranking in the past. Through the analysis of the mean and variance, we show that an optimal rank order is the one that maximizes the overall relevance (mean) of the ranked list at a given risk level (variance). Based on this principle, we then derive an efficient document ranking algorithm. It extends the PRP by considering both the uncertainty of relevance predictions and correlations between retrieved documents. Furthermore, we quantify the benefits of diversification, and theoretically show that diversifying documents is an effective way to reduce the risk of document ranking. Experimental results on the collaborative filtering problem confirms the theoretical insights with improved recommendation performance, e.g., achieved over 300% performance gain over the PRP-based ranking on the user-based recommendation.

AUTHOR =       {Jun Wang},
TITLE =        {“{M}ean-Variance Analysis: A New Document Ranking Theory in Information Retrieval},
BOOKTITLE =    {Proc. of European Conference on Information Retrieval (ECIR 2009)},
YEAR =         {2009}

The ECIR Paper (PDF)

  1. No comments yet.
  1. No trackbacks yet.
You must be logged in to post a comment.