Topic (Query) Selection for IR Evaluation
June 18th, 2009
Comments off
The need for evaluating large amounts of topics (queries) makes
IR evaluation an uneasy task. In this paper, we study a topic se-
lection problem for IR evaluation. The selection criterion is based
on the overall difficulty of the chosen set, as well as the uncertainty
of the final IR metric applied to the systems. Our preliminary ex-
periments demonstrate that our approach helps to identify a set of
topics that provides confident estimates of systems’ performance
while keeping the requirement of the query difficulty.
@INPROCEEDINGS{Wang:sigir2009:3,
AUTHOR = {Jianhan Zhu and Jun Wang and Vishwa Vinay},
TITLE = {Topic (Query) Selection for IR Evaluation},
BOOKTITLE = {SIGIR09 Poster},
YEAR = {2009}
}
