org.kairosdb.bigqueue

Interface IFanOutQueue

All Superinterfaces:

AutoCloseable, Closeable

All Known Implementing Classes:

FanOutQueueImpl

public interface IFanOutQueue
extends Closeable

FanOut queue ADT

Author:

bulldog

Field Summary

Fields

Modifier and Type	Field and Description
static long	EARLIEST Constant represents earliest timestamp
static long	LATEST Constant represents latest timestamp

Method Summary

Modifier and Type	Method and Description
byte[]	dequeue(String fanoutId) Retrieves and removes the front of a fan out queue
byte[]	dequeue(String fanoutId, boolean useLatest) Retrieves and removes the front of a fan out queue
long	enqueue(byte[] data) Adds an item at the back of the queue
long	findClosestIndex(long timestamp) Find an index closest to the specific timestamp when the corresponding item was enqueued.
void	flush() Force to persist current state of the queue, normally, you don't need to flush explicitly since: 1.) FanOutQueue will automatically flush a cached page when it is replaced out, 2.) FanOutQueue uses memory mapped file technology internally, and the OS will flush the changes even your process crashes, call this periodically only if you need transactional reliability and you are aware of the cost to performance.
byte[]	get(long index) Retrieves data item at the specific index of the queue
long	getBackFileSize() Current total size of the back files of this queue
long	getFrontIndex() Get the queue front index, this is the earliest appended index
long	getFrontIndex(String fanoutId) Get front index of specific fanout queue
long	getFrontIndex(String fanoutId, boolean useLatest) Get front index of specific fanout queue
int	getLength(long index) Get length of data item at specific index of the queue
long	getRearIndex() Get the queue rear index, this is the next to be appended index
long	getTimestamp(long index) Get timestamp of data item at specific index of the queue, this is the timestamp when corresponding item was appended into the queue.
boolean	isEmpty() Determines whether the queue is empty
boolean	isEmpty(String fanoutId) Determines whether a fan out queue is empty
boolean	isEmpty(String fanoutId, boolean useLatest) Determines whether a fan out queue is empty
void	limitBackFileSize(long sizeLmit) Limit the back file size of this queue, truncate back files and advance the queue front if necessary.
byte[]	peek(String fanoutId) Peek the item at the front of a fanout queue, without removing it from the queue
byte[]	peek(String fanoutId, boolean useLatest) Peek the item at the front of a fanout queue, without removing it from the queue
int	peekLength(String fanoutId) Peek the length of the item at the front of a fan out queue
int	peekLength(String fanoutId, boolean useLatest) Peek the length of the item at the front of a fan out queue
long	peekTimestamp(String fanoutId) Peek the timestamp of the item at the front of a fan out queue
long	peekTimestamp(String fanoutId, boolean useLatest) Peek the timestamp of the item at the front of a fan out queue
void	removeAll() Removes all items of a queue, this will empty the queue and delete all back data files.
void	removeBefore(long timestamp) Remove all data before specific timestamp, truncate back files and advance the queue front if necessary.
void	resetQueueFrontIndex(String fanoutId, long index) Reset the front index of a fanout queue.
long	size() Total number of items remaining in the queue.
long	size(String fanoutId) Total number of items remaining in the fan out queue
long	size(String fanoutId, boolean useLatest) Total number of items remaining in the fan out queue

Methods inherited from interface java.io.Closeable

close

Field Detail

EARLIEST

static final long EARLIEST

Constant represents earliest timestamp

See Also:

Constant Field Values

LATEST

static final long LATEST

Constant represents latest timestamp

See Also:

Constant Field Values

Method Detail

isEmpty

boolean isEmpty(String fanoutId,
                boolean useLatest)
         throws IOException

Determines whether a fan out queue is empty

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

true if empty, false otherwise

Throws:

IOException - exception thrown if IO error occurs

isEmpty

boolean isEmpty(String fanoutId)
         throws IOException

Determines whether a fan out queue is empty

Parameters:

fanoutId - the fanout identifier

Returns:

true if empty, false otherwise

Throws:

IOException - exception thrown if IO error occurs

isEmpty

boolean isEmpty()

Determines whether the queue is empty

Returns:

true if empty, false otherwise

enqueue

long enqueue(byte[] data)
      throws IOException

Adds an item at the back of the queue

Parameters:

data - to be enqueued data

Returns:

index where the item was appended

Throws:

IOException - exception throws if there is any IO error during enqueue operation.

dequeue

byte[] dequeue(String fanoutId)
        throws IOException

Retrieves and removes the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during dequeue operation.

dequeue

byte[] dequeue(String fanoutId,
               boolean useLatest)
        throws IOException

Retrieves and removes the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during dequeue operation.

peek

byte[] peek(String fanoutId)
     throws IOException

Peek the item at the front of a fanout queue, without removing it from the queue

Parameters:

fanoutId - the fanout identifier

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

peek

byte[] peek(String fanoutId,
            boolean useLatest)
     throws IOException

Peek the item at the front of a fanout queue, without removing it from the queue

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

peekLength

int peekLength(String fanoutId)
        throws IOException

Peek the length of the item at the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

peekLength

int peekLength(String fanoutId,
               boolean useLatest)
        throws IOException

Peek the length of the item at the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

peekTimestamp

long peekTimestamp(String fanoutId)
            throws IOException

Peek the timestamp of the item at the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

peekTimestamp

long peekTimestamp(String fanoutId,
                   boolean useLatest)
            throws IOException

Peek the timestamp of the item at the front of a fan out queue

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

data at the front of a queue

Throws:

IOException - exception throws if there is any IO error during peek operation.

get

byte[] get(long index)
    throws IOException

Retrieves data item at the specific index of the queue

Parameters:

index - data item index

Returns:

data at index

Throws:

IOException - exception throws if there is any IO error during fetch operation.

getLength

int getLength(long index)
       throws IOException

Get length of data item at specific index of the queue

Parameters:

index - data item index

Returns:

length of data item

Throws:

IOException - exception throws if there is any IO error during fetch operation.

getTimestamp

long getTimestamp(long index)
           throws IOException

Get timestamp of data item at specific index of the queue, this is the timestamp when corresponding item was appended into the queue.

Parameters:

index - data item index

Returns:

timestamp of data item

Throws:

IOException - exception throws if there is any IO error during fetch operation.

size

long size(String fanoutId)
   throws IOException

Total number of items remaining in the fan out queue

Parameters:

fanoutId - the fanout identifier

Returns:

total number

Throws:

IOException - exception thrown if IO error occurs

size

long size(String fanoutId,
          boolean useLatest)
   throws IOException

Total number of items remaining in the fan out queue

Parameters:

fanoutId - the fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

total number

Throws:

IOException - exception thrown if IO error occurs

size

long size()

Total number of items remaining in the queue.

Returns:

total number

flush

void flush()

Force to persist current state of the queue, normally, you don't need to flush explicitly since: 1.) FanOutQueue will automatically flush a cached page when it is replaced out, 2.) FanOutQueue uses memory mapped file technology internally, and the OS will flush the changes even your process crashes, call this periodically only if you need transactional reliability and you are aware of the cost to performance.

removeBefore

void removeBefore(long timestamp)
           throws IOException

Remove all data before specific timestamp, truncate back files and advance the queue front if necessary.

Parameters:

timestamp - a timestamp

Throws:

IOException - exception thrown if there was any IO error during the removal operation

limitBackFileSize

void limitBackFileSize(long sizeLmit)
                throws IOException

Limit the back file size of this queue, truncate back files and advance the queue front if necessary. Note, this is a best effort call, exact size limit can't be guaranteed

Parameters:

sizeLmit - size limit

Throws:

IOException - exception thrown if there was any IO error during the operation

getBackFileSize

long getBackFileSize()
              throws IOException

Current total size of the back files of this queue

Returns:

total back file size

Throws:

IOException - exception thrown if there was any IO error during the operation

findClosestIndex

long findClosestIndex(long timestamp)
               throws IOException

Find an index closest to the specific timestamp when the corresponding item was enqueued. to find latest index, use LATEST as timestamp. to find earliest index, use EARLIEST as timestamp.

Parameters:

timestamp - when the corresponding item was appended

Returns:

an index

Throws:

IOException - exception thrown during the operation

resetQueueFrontIndex

void resetQueueFrontIndex(String fanoutId,
                          long index)
                   throws IOException

Reset the front index of a fanout queue.

Parameters:

fanoutId - fanout identifier

index - target index

Throws:

IOException - exception thrown during the operation

removeAll

void removeAll()
        throws IOException

Removes all items of a queue, this will empty the queue and delete all back data files.

Throws:

IOException - exception throws if there is any IO error during dequeue operation.

getFrontIndex

long getFrontIndex()

Get the queue front index, this is the earliest appended index

Returns:

an index

getFrontIndex

long getFrontIndex(String fanoutId)
            throws IOException

Get front index of specific fanout queue

Parameters:

fanoutId - fanout identifier

Returns:

an index

Throws:

IOException - exception thrown if IO error occurs

getFrontIndex

long getFrontIndex(String fanoutId,
                   boolean useLatest)
            throws IOException

Get front index of specific fanout queue

Parameters:

fanoutId - fanout identifier

useLatest - if no offset has been recorded the head of the queue is used

Returns:

an index

Throws:

IOException - exception thrown if IO error occurs

getRearIndex

long getRearIndex()

Get the queue rear index, this is the next to be appended index

Returns:

an index