The TREC (Text REtrieval Conference) conferences, of which there have been two, with the third due to start early 1994, are concerned with controlled comparisons of different methods of retrieving documents from large collections of assorted textual material. They are funded by the US Advanced Projects Research Agency (ARPA) and organised by Donna Harman of NIST (National Institute for Standards and Technology). There were about 31 participants, academic and commercial, in the TREC-2 conference which took place at Gaithersburg, MD in September 1993 [2]. Information needs are presented in the form of highly structured ``topics'' from which queries are to be derived automatically and/or manually by participants. Documents include newspaper articles, entries from the Federal Register, patents and technical abstracts, varying in length from a line or two to several hundred thousand words.
A large number of relevance judgments have been made at NIST by a panel of experts assessing the top-ranked documents retrieved by some of the participants in TREC--1 and TREC--2. The number of known relevant documents for the 150 topics varies between 1 and more than 1000, with a mean of 281.