org.galagosearch.core.eval
Class Main

java.lang.Object
  org.galagosearch.core.eval.Main

public class Main
extends java.lang.Object

Author:

trevor

Constructor Summary

Main()

Method Summary

static void	comparisonEvaluation(SetRetrievalEvaluator baseline, SetRetrievalEvaluator treatment, boolean useRandomized) Compares two ranked lists with statistical tests on most major metrics.
static SetRetrievalEvaluator	create(java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Document>> allRankings, java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Judgment>> allJudgments) Creates a SetRetrievalEvaluator from data from loadRanking and loadJudgments.
static java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Judgment>>	loadJudgments(java.lang.String filename) Loads a TREC judgments file.
static java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Document>>	loadRanking(java.lang.String filename) Reads in a TREC ranking file.
static void	main(java.lang.String[] args)
static void	singleEvaluation(SetRetrievalEvaluator setEvaluator) When run as a standalone application, this returns output very similar to that of trec_eval.
static void	usage()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Main

public Main()

Method Detail

loadJudgments

public static java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Judgment>> loadJudgments(java.lang.String filename)
                                                                                                          throws java.io.IOException,
                                                                                                                 java.io.FileNotFoundException

Loads a TREC judgments file.

Parameters:

filename - The filename of the judgments file to load.

Returns:

Maps from query numbers to lists of judgments for each query.

Throws:

java.io.IOException

java.io.FileNotFoundException

loadRanking

public static java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Document>> loadRanking(java.lang.String filename)
                                                                                                        throws java.io.IOException,
                                                                                                               java.io.FileNotFoundException

Reads in a TREC ranking file.

Parameters:

filename - The filename of the ranking file.

Returns:

A map from query numbers to document ranking lists.

Throws:

java.io.IOException

java.io.FileNotFoundException

create

public static SetRetrievalEvaluator create(java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Document>> allRankings,
                                           java.util.TreeMap<java.lang.String,java.util.ArrayList<RetrievalEvaluator.Judgment>> allJudgments)

Creates a SetRetrievalEvaluator from data from loadRanking and loadJudgments.

singleEvaluation

public static void singleEvaluation(SetRetrievalEvaluator setEvaluator)

When run as a standalone application, this returns output very similar to that of trec_eval. The first argument is the ranking file, and the second argument is the judgments file, both in standard TREC format.

comparisonEvaluation

public static void comparisonEvaluation(SetRetrievalEvaluator baseline,
                                        SetRetrievalEvaluator treatment,
                                        boolean useRandomized)

Compares two ranked lists with statistical tests on most major metrics.

usage

public static void usage()

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

Throws:

java.io.IOException