org.galagosearch.core.index
Class IndexWriter

java.lang.Object
  org.galagosearch.core.index.IndexWriter

public class IndexWriter
extends java.lang.Object

This class writes index files, which are used for most Galago indexes. An index is a mapping between a key and a value, much like a TreeMap. The keys are sorted to allow iteration over the whole file. Keys are stored using prefix compression to save space. The structure is designed for fast random access on disk. For indexes, we assume that the data in each value is already compressed, so IndexWriter does no additional compression. However, if the isCompressed flag is set, IndexWriter will compress the value data. This is convenient for storing documents in an index. Keys cannot be longer than 256 bytes, and they must be added in sorted order.

Author:

trevor

Field Summary

static long

MAGIC_NUMBER

Constructor Summary

IndexWriter(java.lang.String outputFilename)

IndexWriter(java.lang.String outputFilename, org.galagosearch.tupleflow.Parameters parameters)
Creates a new instance of IndexWriter

IndexWriter(org.galagosearch.tupleflow.TupleFlowParameters parameters)

Method Summary

void	add(IndexElement list)
long	bufferedSize() Gives a conservative estimate of the buffered size of the data, excluding the most recent inverted list.
void	close()
void	flush() Flush all lists out to disk.
long	getBlockSize()
org.galagosearch.tupleflow.Parameters	getManifest() Returns the current copy of the manifest, which will be stored in the completed index file.
void	updateBufferedSize(IndexElement list)
boolean	wordsInOrder(java.util.List<IndexElement> blockLists) Returns true if the lists are sorted in ascending order by key.
void	writeBlock(java.util.List<IndexElement> blockLists, long length)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAGIC_NUMBER

public static final long MAGIC_NUMBER

See Also:

Constant Field Values

Constructor Detail

IndexWriter

public IndexWriter(java.lang.String outputFilename,
                   org.galagosearch.tupleflow.Parameters parameters)
            throws java.io.FileNotFoundException,
                   java.io.IOException

Creates a new instance of IndexWriter

Throws:

java.io.FileNotFoundException

java.io.IOException

IndexWriter

public IndexWriter(java.lang.String outputFilename)
            throws java.io.FileNotFoundException,
                   java.io.IOException

Throws:

java.io.FileNotFoundException

java.io.IOException

IndexWriter

public IndexWriter(org.galagosearch.tupleflow.TupleFlowParameters parameters)
            throws java.io.FileNotFoundException,
                   java.io.IOException

Throws:

java.io.FileNotFoundException

java.io.IOException

Method Detail

getManifest

public org.galagosearch.tupleflow.Parameters getManifest()

Returns the current copy of the manifest, which will be stored in the completed index file. This data is not written until close() is called.

bufferedSize

public long bufferedSize()

Gives a conservative estimate of the buffered size of the data, excluding the most recent inverted list. Does not include savings due to key overlap compression.

updateBufferedSize

public void updateBufferedSize(IndexElement list)

flush

public void flush()
           throws java.io.IOException

Flush all lists out to disk.

Throws:

java.io.IOException

getBlockSize

public long getBlockSize()

wordsInOrder

public boolean wordsInOrder(java.util.List<IndexElement> blockLists)

Returns true if the lists are sorted in ascending order by key.

Parameters:

blockLists -

writeBlock

public void writeBlock(java.util.List<IndexElement> blockLists,
                       long length)
                throws java.io.IOException

Throws:

java.io.IOException

add

public void add(IndexElement list)
         throws java.io.IOException

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException