Package org.rocksdb

Class BlockBasedTableConfig

  • public class BlockBasedTableConfig
extends TableFormatConfig
    The config for plain table sst format. BlockBasedTable is a RocksDB's default SST file format.

    • Constructor Detail

      • BlockBasedTableConfig

        public BlockBasedTableConfig()

    • Method Detail

      • cacheIndexAndFilterBlocks

        public boolean cacheIndexAndFilterBlocks()
        Indicating if we'd put index/filter blocks to the block cache. If not specified, each "table reader" object will pre-load index/filter block during table initialization.
        Returns:
        if index and filter blocks should be put in block cache.

      • setCacheIndexAndFilterBlocks

        public BlockBasedTableConfig setCacheIndexAndFilterBlocks​(boolean cacheIndexAndFilterBlocks)
        Indicating if we'd put index/filter blocks to the block cache. If not specified, each "table reader" object will pre-load index/filter block during table initialization.
        Parameters:
        cacheIndexAndFilterBlocks - and filter blocks should be put in block cache.
        Returns:
        the reference to the current config.

      • cacheIndexAndFilterBlocksWithHighPriority

        public boolean cacheIndexAndFilterBlocksWithHighPriority()
        Indicates if index and filter blocks will be treated as high-priority in the block cache. See note below about applicability. If not specified, defaults to true.
        Returns:
        if index and filter blocks will be treated as high-priority.

      • setCacheIndexAndFilterBlocksWithHighPriority

        public BlockBasedTableConfig setCacheIndexAndFilterBlocksWithHighPriority​(boolean cacheIndexAndFilterBlocksWithHighPriority)
        If true, cache index and filter blocks with high priority. If set to true, depending on implementation of block cache, index and filter blocks may be less likely to be evicted than data blocks.
        Parameters:
        cacheIndexAndFilterBlocksWithHighPriority - if index and filter blocks will be treated as high-priority.
        Returns:
        the reference to the current config.

      • pinL0FilterAndIndexBlocksInCache

        public boolean pinL0FilterAndIndexBlocksInCache()
        Indicating if we'd like to pin L0 index/filter blocks to the block cache. If not specified, defaults to false.
        Returns:
        if L0 index and filter blocks should be pinned to the block cache.

      • setPinL0FilterAndIndexBlocksInCache

        public BlockBasedTableConfig setPinL0FilterAndIndexBlocksInCache​(boolean pinL0FilterAndIndexBlocksInCache)
        Indicating if we'd like to pin L0 index/filter blocks to the block cache. If not specified, defaults to false.
        Parameters:
        pinL0FilterAndIndexBlocksInCache - pin blocks in block cache
        Returns:
        the reference to the current config.

      • pinTopLevelIndexAndFilter

        public boolean pinTopLevelIndexAndFilter()
        Indicates if top-level index and filter blocks should be pinned.
        Returns:
        if top-level index and filter blocks should be pinned.

      • setPinTopLevelIndexAndFilter

        public BlockBasedTableConfig setPinTopLevelIndexAndFilter​(boolean pinTopLevelIndexAndFilter)
        If cacheIndexAndFilterBlocks is true and the below is true, then the top-level index of partitioned filter and index blocks are stored in the cache, but a reference is held in the "table reader" object so the blocks are pinned and only evicted from cache when the table reader is freed. This is not limited to l0 in LSM tree.
        Parameters:
        pinTopLevelIndexAndFilter - if top-level index and filter blocks should be pinned.
        Returns:
        the reference to the current config.

      • indexType

        public IndexType indexType()
        Get the index type.
        Returns:
        the currently set index type

      • setIndexType

        public BlockBasedTableConfig setIndexType​(IndexType indexType)
        Sets the index type to used with this table.
        Parameters:
        indexType - IndexType value
        Returns:
        the reference to the current option.

      • dataBlockIndexType

        public DataBlockIndexType dataBlockIndexType()
        Get the data block index type.
        Returns:
        the currently set data block index type

      • checksumType

        public ChecksumType checksumType()
        Get the checksum type to be used with this table.
        Returns:
        the currently set checksum type

      • noBlockCache

        public boolean noBlockCache()
        Determine if the block cache is disabled.
        Returns:
        if block cache is disabled

      • setNoBlockCache

        public BlockBasedTableConfig setNoBlockCache​(boolean noBlockCache)
        Disable block cache. If this is set to true, then no block cache should be used, and the setBlockCache(Cache) should point to a null object. Default: false
        Parameters:
        noBlockCache - if use block cache
        Returns:
        the reference to the current config.

      • setBlockCache

        public BlockBasedTableConfig setBlockCache​(Cache blockCache)
        Use the specified cache for blocks. When not null this take precedence even if the user sets a block cache size. Cache should not be disposed before options instances using this cache is disposed. Cache instance can be re-used in multiple options instances.
        Parameters:
        blockCache - Cache Cache java instance (e.g. LRUCache).
        Returns:
        the reference to the current config.

      • setPersistentCache

        public BlockBasedTableConfig setPersistentCache​(PersistentCache persistentCache)
        Use the specified persistent cache. If !null use the specified cache for pages read from device, otherwise no page cache is used.
        Parameters:
        persistentCache - the persistent cache
        Returns:
        the reference to the current config.

      • setBlockCacheCompressed

        public BlockBasedTableConfig setBlockCacheCompressed​(Cache blockCacheCompressed)
        Use the specified cache for compressed blocks. If null, RocksDB will not use a compressed block cache. Note: though it looks similar to setBlockCache(Cache), RocksDB doesn't put the same type of object there. Cache should not be disposed before options instances using this cache is disposed. Cache instance can be re-used in multiple options instances.
        Parameters:
        blockCacheCompressed - Cache Cache java instance (e.g. LRUCache).
        Returns:
        the reference to the current config.

      • blockSize

        public long blockSize()
        Get the approximate size of user data packed per block.
        Returns:
        block size in bytes

      • setBlockSize

        public BlockBasedTableConfig setBlockSize​(long blockSize)
        Approximate size of user data packed per block. Note that the block size specified here corresponds to uncompressed data. The actual size of the unit read from disk may be smaller if compression is enabled. This parameter can be changed dynamically. Default: 4K
        Parameters:
        blockSize - block size in bytes
        Returns:
        the reference to the current config.

      • blockSizeDeviation

        public int blockSizeDeviation()
        Returns:
        the hash table ratio.

      • setBlockSizeDeviation

        public BlockBasedTableConfig setBlockSizeDeviation​(int blockSizeDeviation)
        This is used to close a block before it reaches the configured blockSize(). If the percentage of free space in the current block is less than this specified number and adding a new record to the block will exceed the configured block size, then this block will be closed and the new record will be written to the next block. Default is 10.
        Parameters:
        blockSizeDeviation - the deviation to block size allowed
        Returns:
        the reference to the current config.

      • blockRestartInterval

        public int blockRestartInterval()
        Get the block restart interval.
        Returns:
        block restart interval

      • setBlockRestartInterval

        public BlockBasedTableConfig setBlockRestartInterval​(int restartInterval)
        Set the block restart interval.
        Parameters:
        restartInterval - block restart interval.
        Returns:
        the reference to the current config.

      • indexBlockRestartInterval

        public int indexBlockRestartInterval()
        Get the index block restart interval.
        Returns:
        index block restart interval

      • setIndexBlockRestartInterval

        public BlockBasedTableConfig setIndexBlockRestartInterval​(int restartInterval)
        Set the index block restart interval
        Parameters:
        restartInterval - index block restart interval.
        Returns:
        the reference to the current config.

      • metadataBlockSize

        public long metadataBlockSize()
        Get the block size for partitioned metadata.
        Returns:
        block size for partitioned metadata.

      • setMetadataBlockSize

        public BlockBasedTableConfig setMetadataBlockSize​(long metadataBlockSize)
        Set block size for partitioned metadata.
        Parameters:
        metadataBlockSize - Partitioned metadata block size.
        Returns:
        the reference to the current config.

      • partitionFilters

        public boolean partitionFilters()
        Indicates if we're using partitioned filters.
        Returns:
        if we're using partition filters.

      • setPartitionFilters

        public BlockBasedTableConfig setPartitionFilters​(boolean partitionFilters)
        Use partitioned full filters for each SST file. This option is incompatible with block-based filters. Defaults to false.
        Parameters:
        partitionFilters - use partition filters.
        Returns:
        the reference to the current config.

      • optimizeFiltersForMemory

        @Experimental("Option to generate Bloom filters that minimize memory internal fragmentation")
public boolean optimizeFiltersForMemory()
        Option to generate Bloom filters that minimize memory internal fragmentation. See setOptimizeFiltersForMemory(boolean).
        Returns:
        true if bloom filters are used to minimize memory internal fragmentation

      • setOptimizeFiltersForMemory

        @Experimental("Option to generate Bloom filters that minimize memory internal fragmentation")
public BlockBasedTableConfig setOptimizeFiltersForMemory​(boolean optimizeFiltersForMemory)
        Option to generate Bloom filters that minimize memory internal fragmentation. When false, malloc_usable_size is not available, or format_version < 5, filters are generated without regard to internal fragmentation when loaded into memory (historical behavior). When true (and malloc_usable_size is available and formatVersion() >= 5), then Bloom filters are generated to "round up" and "round down" their sizes to minimize internal fragmentation when loaded into memory, assuming the reading DB has the same memory allocation characteristics as the generating DB. This option does not break forward or backward compatibility. While individual filters will vary in bits/key and false positive rate when setting is true, the implementation attempts to maintain a weighted average FP rate for filters consistent with this option set to false. With Jemalloc for example, this setting is expected to save about 10% of the memory footprint and block cache charge of filters, while increasing disk usage of filters by about 1-2% due to encoding efficiency losses with variance in bits/key. NOTE: Because some memory counted by block cache might be unmapped pages within internal fragmentation, this option can increase observed RSS memory usage. With cacheIndexAndFilterBlocks() == true, this option makes the block cache better at using space it is allowed. NOTE: Do not set to true if you do not trust malloc_usable_size. With this option, RocksDB might access an allocated memory object beyond its original size if malloc_usable_size says it is safe to do so. While this can be considered bad practice, it should not produce undefined behavior unless malloc_usable_size is buggy or broken.
        Parameters:
        optimizeFiltersForMemory - true to enable Bloom filters that minimize memory internal fragmentation, or false to disable.
        Returns:
        the reference to the current config.

      • useDeltaEncoding

        public boolean useDeltaEncoding()
        Determine if delta encoding is being used to compress block keys.
        Returns:
        true if delta encoding is enabled, false otherwise.

      • setUseDeltaEncoding

        public BlockBasedTableConfig setUseDeltaEncoding​(boolean useDeltaEncoding)
        Use delta encoding to compress keys in blocks. NOTE: ReadOptions.pinData() requires this option to be disabled. Default: true
        Parameters:
        useDeltaEncoding - true to enable delta encoding
        Returns:
        the reference to the current config.

      • filterPolicy

        public Filter filterPolicy()
        Get the filter policy.
        Returns:
        the current filter policy.

      • setFilterPolicy

        public BlockBasedTableConfig setFilterPolicy​(Filter filterPolicy)
        Use the specified filter policy to reduce disk reads. Filter should not be disposed before options instances using this filter is disposed. If AbstractNativeReference.dispose() function is not called, then filter object will be GC'd automatically. Filter instance can be re-used in multiple options instances.
        Parameters:
        filterPolicy - Filter Filter Policy java instance.
        Returns:
        the reference to the current config.

      • wholeKeyFiltering

        public boolean wholeKeyFiltering()
        Determine if whole keys as opposed to prefixes are placed in the filter.
        Returns:
        if whole key filtering is enabled

      • setWholeKeyFiltering

        public BlockBasedTableConfig setWholeKeyFiltering​(boolean wholeKeyFiltering)
        If true, place whole keys in the filter (not just prefixes). This must generally be true for gets to be efficient. Default: true
        Parameters:
        wholeKeyFiltering - if enable whole key filtering
        Returns:
        the reference to the current config.

      • verifyCompression

        public boolean verifyCompression()
        Returns true when compression verification is enabled. See setVerifyCompression(boolean).
        Returns:
        true if compression verification is enabled.

      • setVerifyCompression

        public BlockBasedTableConfig setVerifyCompression​(boolean verifyCompression)
        Verify that decompressing the compressed block gives back the input. This is a verification mode that we use to detect bugs in compression algorithms.
        Parameters:
        verifyCompression - true to enable compression verification.
        Returns:
        the reference to the current config.

      • readAmpBytesPerBit

        public int readAmpBytesPerBit()
        Get the Read amplification bytes per-bit. See setReadAmpBytesPerBit(int).
        Returns:
        the bytes per-bit.

      • setReadAmpBytesPerBit

        public BlockBasedTableConfig setReadAmpBytesPerBit​(int readAmpBytesPerBit)
        Set the Read amplification bytes per-bit. If used, For every data block we load into memory, we will create a bitmap of size ((block_size / `read_amp_bytes_per_bit`) / 8) bytes. This bitmap will be used to figure out the percentage we actually read of the blocks. When this feature is used Tickers::READ_AMP_ESTIMATE_USEFUL_BYTES and Tickers::READ_AMP_TOTAL_READ_BYTES can be used to calculate the read amplification using this formula (READ_AMP_TOTAL_READ_BYTES / READ_AMP_ESTIMATE_USEFUL_BYTES) value => memory usage (percentage of loaded blocks memory) 1 => 12.50 % 2 => 06.25 % 4 => 03.12 % 8 => 01.56 % 16 => 00.78 % Note: This number must be a power of 2, if not it will be sanitized to be the next lowest power of 2, for example a value of 7 will be treated as 4, a value of 19 will be treated as 16. Default: 0 (disabled)
        Parameters:
        readAmpBytesPerBit - the bytes per-bit
        Returns:
        the reference to the current config.

      • formatVersion

        public int formatVersion()
        Get the format version. See setFormatVersion(int).
        Returns:
        the currently configured format version.

      • setFormatVersion

        public BlockBasedTableConfig setFormatVersion​(int formatVersion)

        We currently have five versions:

        • 0 - This version is currently written out by all RocksDB's versions by default. Can be read by really old RocksDB's. Doesn't support changing checksum (default is CRC32).
        • 1 - Can be read by RocksDB's versions since 3.0. Supports non-default checksum, like xxHash. It is written by RocksDB when BlockBasedTableOptions::checksum is something other than kCRC32c. (version 0 is silently upconverted)
        • 2 - Can be read by RocksDB's versions since 3.10. Changes the way we encode compressed blocks with LZ4, BZip2 and Zlib compression. If you don't plan to run RocksDB before version 3.10, you should probably use this.
        • 3 - Can be read by RocksDB's versions since 5.15. Changes the way we encode the keys in index blocks. If you don't plan to run RocksDB before version 5.15, you should probably use this. This option only affects newly written tables. When reading existing tables, the information about version is read from the footer.
        • 4 - Can be read by RocksDB's versions since 5.16. Changes the way we encode the values in index blocks. If you don't plan to run RocksDB before version 5.16 and you are using index_block_restart_interval > 1, you should probably use this as it would reduce the index size. This option only affects newly written tables. When reading existing tables, the information about version is read from the footer.
        • 5 - Can be read by RocksDB's versions since 6.6.0. Full and partitioned filters use a generally faster and more accurate Bloom filter implementation, with a different schema.
        Parameters:
        formatVersion - integer representing the version to be used.
        Returns:
        the reference to the current option.

      • enableIndexCompression

        public boolean enableIndexCompression()
        Determine if index compression is enabled. See setEnableIndexCompression(boolean).
        Returns:
        true if index compression is enabled, false otherwise

      • setEnableIndexCompression

        public BlockBasedTableConfig setEnableIndexCompression​(boolean enableIndexCompression)
        Store index blocks on disk in compressed format. Changing this option to false will avoid the overhead of decompression if index blocks are evicted and read back.
        Parameters:
        enableIndexCompression - true to enable index compression, false to disable
        Returns:
        the reference to the current option.

      • blockAlign

        public boolean blockAlign()
        Determines whether data blocks are aligned on the lesser of page size and block size.
        Returns:
        true if data blocks are aligned on the lesser of page size and block size.

      • setBlockAlign

        public BlockBasedTableConfig setBlockAlign​(boolean blockAlign)
        Set whether data blocks should be aligned on the lesser of page size and block size.
        Parameters:
        blockAlign - true to align data blocks on the lesser of page size and block size.
        Returns:
        the reference to the current option.

      • indexShortening

        public IndexShorteningMode indexShortening()
        Get the index shortening mode.
        Returns:
        the index shortening mode.

      • blockCacheSize

        @Deprecated
public long blockCacheSize()
        Deprecated.
        Get the size of the cache in bytes that will be used by RocksDB.
        Returns:
        block cache size in bytes

      • setBlockCacheSize

        @Deprecated
public BlockBasedTableConfig setBlockCacheSize​(long blockCacheSize)
        Deprecated.
        Use setBlockCache(Cache).
        Set the size of the cache in bytes that will be used by RocksDB. If cacheSize is negative, then cache will not be used. DEFAULT: 8M
        Parameters:
        blockCacheSize - block cache size in bytes
        Returns:
        the reference to the current config.

      • cacheNumShardBits

        @Deprecated
public int cacheNumShardBits()
        Deprecated.
        Returns the number of shard bits used in the block cache. The resulting number of shards would be 2 ^ (returned value). Any negative number means use default settings.
        Returns:
        the number of shard bits used in the block cache.

      • setCacheNumShardBits

        @Deprecated
public BlockBasedTableConfig setCacheNumShardBits​(int blockCacheNumShardBits)
        Deprecated.
        Use setBlockCache(Cache).
        Controls the number of shards for the block cache. This is applied only if cacheSize is set to non-negative.
        Parameters:
        blockCacheNumShardBits - the number of shard bits. The resulting number of shards would be 2 ^ numShardBits. Any negative number means use default settings."
        Returns:
        the reference to the current option.

      • blockCacheCompressedSize

        @Deprecated
public long blockCacheCompressedSize()
        Deprecated.
        Size of compressed block cache. If 0, then block_cache_compressed is set to null.
        Returns:
        size of compressed block cache.

      • setBlockCacheCompressedSize

        @Deprecated
public BlockBasedTableConfig setBlockCacheCompressedSize​(long blockCacheCompressedSize)
        Deprecated.
        Use setBlockCacheCompressed(Cache).
        Size of compressed block cache. If 0, then block_cache_compressed is set to null.
        Parameters:
        blockCacheCompressedSize - of compressed block cache.
        Returns:
        the reference to the current config.

      • blockCacheCompressedNumShardBits

        @Deprecated
public int blockCacheCompressedNumShardBits()
        Deprecated.
        Controls the number of shards for the block compressed cache. This is applied only if blockCompressedCacheSize is set to non-negative.
        Returns:
        numShardBits the number of shard bits. The resulting number of shards would be 2 ^ numShardBits. Any negative number means use default settings.

      • setBlockCacheCompressedNumShardBits

        @Deprecated
public BlockBasedTableConfig setBlockCacheCompressedNumShardBits​(int blockCacheCompressedNumShardBits)
        Deprecated.
        Use setBlockCacheCompressed(Cache).
        Controls the number of shards for the block compressed cache. This is applied only if blockCompressedCacheSize is set to non-negative.
        Parameters:
        blockCacheCompressedNumShardBits - the number of shard bits. The resulting number of shards would be 2 ^ numShardBits. Any negative number means use default settings."
        Returns:
        the reference to the current option.

      • hashIndexAllowCollision

        @Deprecated
public boolean hashIndexAllowCollision()
        Deprecated.
        This option is now deprecated. No matter what value it is set to, it will behave as if hashIndexAllowCollision() == true.
        Influence the behavior when kHashSearch is used. if false, stores a precise prefix to block range mapping if true, does not store prefix and allows prefix hash collision (less memory consumption)
        Returns:
        if hash collisions should be allowed.

      • setHashIndexAllowCollision

        @Deprecated
public BlockBasedTableConfig setHashIndexAllowCollision​(boolean hashIndexAllowCollision)
        Deprecated.
        This option is now deprecated. No matter what value it is set to, it will behave as if hashIndexAllowCollision() == true.
        Influence the behavior when kHashSearch is used. if false, stores a precise prefix to block range mapping if true, does not store prefix and allows prefix hash collision (less memory consumption)
        Parameters:
        hashIndexAllowCollision - points out if hash collisions should be allowed.
        Returns:
        the reference to the current config.

      • newTableFactoryHandle

        protected long newTableFactoryHandle()
        Description copied from class: TableFormatConfig

        This function should only be called by Options.setTableFormatConfig(), which will create a c++ shared-pointer to the c++ TableFactory that associated with the Java TableFormatConfig.

        Specified by:
        newTableFactoryHandle in class TableFormatConfig
        Returns:
        native handle address to native table instance.