io.warp10.continuum.gts

Class GTSSplitter

java.lang.Object

io.warp10.continuum.gts.GTSSplitter

public class GTSSplitter
extends java.lang.Object

Class dedicated to splitting a (supposedly large) GTS into smaller ones.

Constructor Summary

Constructors

Constructor and Description
GTSSplitter()

Method Summary

Modifier and Type	Method and Description
static java.util.List<GTSWrapper>	chunk(GTSDecoder decoder, long clipfrom, long clipto, long tsboundary, long chunkwidth, java.lang.String chunklabel, java.lang.String fileprefix, long maxsize, long maxgts, float lwmratio) Split a GTSDecoder into multiple GTSWrapper instances, spilling on disk if necessary.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GTSSplitter

public GTSSplitter()

Method Detail

chunk

public static java.util.List<GTSWrapper> chunk(GTSDecoder decoder,
                                               long clipfrom,
                                               long clipto,
                                               long tsboundary,
                                               long chunkwidth,
                                               java.lang.String chunklabel,
                                               java.lang.String fileprefix,
                                               long maxsize,
                                               long maxgts,
                                               float lwmratio)
                                        throws WarpScriptException,
                                               java.io.IOException

Split a GTSDecoder into multiple GTSWrapper instances, spilling on disk if necessary. The input GTSDecoder will be clipped to ['clipfrom','clipto']. The output GTSWrapper will be bucketized with each a single bucket of 'chunkwidth' span. All buckets within ['clipfrom','clipto'] will have a resulting GTSWrapper. Chunk ends (lastbucket of each resulting GTSWrapper) will be congruent to 'tsboundary' modulo 'chunkwidth'

Parameters:

decoder - GTSDecoder to split

clipfrom - Earliest timestamp of clipping region (included)

clipto - Latest timestamp of clipping region (included)

tsboundary - Timestamp boundary of the end of each chunk

chunkwidth - Width of each chunk

chunklabel - Name of the chunk label

fileprefix - Prefix of the target files, the suffix '-' + chunkend will be appended

maxsize - Maximum size in bytes the encoders can occupy in memory.

maxgts - Maximum number of encoders at any given time

lwmratio - When flushing encoders to disk, we will attempt to lower size or number of gts below lwmratio * {maxsize,maxgts}

Returns:

a list of GTSEncoders or null if spilling on disk happened

Throws:

WarpScriptException

java.io.IOException