org.rdfhdt.hdt.dictionary.impl.section

Class PFCDictionarySectionBig

java.lang.Object

org.rdfhdt.hdt.dictionary.impl.section.PFCDictionarySectionBig

All Implemented Interfaces:

Closeable, AutoCloseable, DictionarySection, DictionarySectionPrivate

public class PFCDictionarySectionBig
extends Object
implements DictionarySectionPrivate

Implementation of Plain Front Coding that divides the data in different arrays, therefore overcoming Java's limitation of 2Gb for a single array. It allows loading much bigger files, but waste some memory in pointers to the blocks and some CPU to locate the array at search time.

Author:

mario.arias, Lyudmila Balakireva

Field Summary

Fields

Modifier and Type	Field and Description
static int	BLOCK_PER_BUFFER
protected SequenceLog64Big	blocks
protected int	blocksize
static int	DEFAULT_BLOCK_SIZE
protected int	numstrings
protected long	size
static int	TYPE_INDEX

Constructor Summary

Constructors

Constructor and Description
PFCDictionarySectionBig(HDTOptions spec)

Method Summary

Modifier and Type	Method and Description
void	close()
CharSequence	extract(long id)
long	getNumberOfElements()
Iterator<CharSequence>	getSortedEntries()
void	load(InputStream input, ProgressListener listener) Load dictionary section from a stream.
void	load(Iterator<CharSequence> it, long numentries, ProgressListener listener)
void	load(TempDictionarySection other, ProgressListener listener) Load entries from another dictionary section.
long	locate(CharSequence str)
protected int	locateBlock(CharSequence str) Locate the block of a string doing binary search.
protected int	locateInBlock(long blocknum, CharSequence str)
void	save(OutputStream output, ProgressListener listener) Serialize dictionary section to a stream.
long	size()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TYPE_INDEX

public static final int TYPE_INDEX

See Also:

Constant Field Values

DEFAULT_BLOCK_SIZE

public static final int DEFAULT_BLOCK_SIZE

See Also:

Constant Field Values

BLOCK_PER_BUFFER

public static final int BLOCK_PER_BUFFER

See Also:

Constant Field Values

blocks

protected SequenceLog64Big blocks

blocksize

protected int blocksize

numstrings

protected int numstrings

size

protected long size

Constructor Detail

PFCDictionarySectionBig

public PFCDictionarySectionBig(HDTOptions spec)

Method Detail

load

public void load(TempDictionarySection other,
                 ProgressListener listener)

Description copied from interface: DictionarySectionPrivate

Load entries from another dictionary section.

Specified by:

load in interface DictionarySectionPrivate

load

public void load(Iterator<CharSequence> it,
                 long numentries,
                 ProgressListener listener)

locateBlock

protected int locateBlock(CharSequence str)

Locate the block of a string doing binary search.

locate

public long locate(CharSequence str)

Specified by:

locate in interface DictionarySection

locateInBlock

protected int locateInBlock(long blocknum,
                            CharSequence str)

extract

public CharSequence extract(long id)

Specified by:

extract in interface DictionarySection

size

public long size()

Specified by:

size in interface DictionarySection

getNumberOfElements

public long getNumberOfElements()

Specified by:

getNumberOfElements in interface DictionarySection

getSortedEntries

public Iterator<CharSequence> getSortedEntries()

Specified by:

getSortedEntries in interface DictionarySection

save

public void save(OutputStream output,
                 ProgressListener listener)
          throws IOException

Description copied from interface: DictionarySectionPrivate

Serialize dictionary section to a stream.

Specified by:

save in interface DictionarySectionPrivate

Throws:

IOException

load

public void load(InputStream input,
                 ProgressListener listener)
          throws IOException

Description copied from interface: DictionarySectionPrivate

Load dictionary section from a stream.

Specified by:

load in interface DictionarySectionPrivate

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException