io.druid.segment.data

Class CompressionFactory

java.lang.Object

io.druid.segment.data.CompressionFactory

public class CompressionFactory
extends Object

Compression of metrics is done by using a combination of CompressedObjectStrategy.CompressionStrategy and Encoding(such as CompressionFactory.LongEncodingStrategy for type Long). CompressionStrategy is unaware of the data type and is based on byte operations. It must compress and decompress in block of bytes. Encoding refers to compression method relies on data format, so a different set of Encodings exist for each data type.

Storage Format : Byte 1 : version (currently 0x02) Byte 2 - 5 : number of values Byte 6 - 9 : size per block (even if block format isn't used, this is needed for backward compatibility) Byte 10 : compression strategy (contains a flag if there's an encoding byte, see below for how the flag is defined) Byte 11(optional) : encoding type

Encoding specific header (described below)

Block related header (if block compression is used, described in GenericIndexed)

Values

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CompressionFactory.LongEncodingFormat
static interface	CompressionFactory.LongEncodingReader
static class	CompressionFactory.LongEncodingStrategy
static interface	CompressionFactory.LongEncodingWriter This writer output encoded values to the given ByteBuffer or OutputStream.

Field Summary

Fields

Modifier and Type	Field and Description
static CompressionFactory.LongEncodingStrategy	DEFAULT_LONG_ENCODING_STRATEGY
static byte	DELTA_ENCODING_VERSION
static CompressionFactory.LongEncodingFormat	LEGACY_LONG_ENCODING_FORMAT
static int	MAX_TABLE_SIZE
static byte	TABLE_ENCODING_VERSION

Method Summary

Modifier and Type	Method and Description
static byte	clearEncodingFlag(byte strategyId)
static FloatSupplierSerializer	getFloatSerializer(IOPeon ioPeon, String filenameBase, ByteOrder order, CompressedObjectStrategy.CompressionStrategy compressionStrategy)
static com.google.common.base.Supplier<IndexedFloats>	getFloatSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressedObjectStrategy.CompressionStrategy strategy, SmooshedFileMapper fileMapper)
static LongSupplierSerializer	getLongSerializer(IOPeon ioPeon, String filenameBase, ByteOrder order, CompressionFactory.LongEncodingStrategy encodingStrategy, CompressedObjectStrategy.CompressionStrategy compressionStrategy)
static com.google.common.base.Supplier<IndexedLongs>	getLongSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionFactory.LongEncodingFormat encodingFormat, CompressedObjectStrategy.CompressionStrategy strategy, SmooshedFileMapper fileMapper)
static boolean	hasEncodingFlag(byte strategyId)
static byte	setEncodingFlag(byte strategyId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_LONG_ENCODING_STRATEGY

public static final CompressionFactory.LongEncodingStrategy DEFAULT_LONG_ENCODING_STRATEGY

LEGACY_LONG_ENCODING_FORMAT

public static final CompressionFactory.LongEncodingFormat LEGACY_LONG_ENCODING_FORMAT

DELTA_ENCODING_VERSION

public static final byte DELTA_ENCODING_VERSION

See Also:

Constant Field Values

TABLE_ENCODING_VERSION

public static final byte TABLE_ENCODING_VERSION

See Also:

Constant Field Values

MAX_TABLE_SIZE

public static final int MAX_TABLE_SIZE

See Also:

Constant Field Values

Method Detail

hasEncodingFlag

public static boolean hasEncodingFlag(byte strategyId)

setEncodingFlag

public static byte setEncodingFlag(byte strategyId)

clearEncodingFlag

public static byte clearEncodingFlag(byte strategyId)

getLongSupplier

public static com.google.common.base.Supplier<IndexedLongs> getLongSupplier(int totalSize,
                                                                            int sizePer,
                                                                            ByteBuffer fromBuffer,
                                                                            ByteOrder order,
                                                                            CompressionFactory.LongEncodingFormat encodingFormat,
                                                                            CompressedObjectStrategy.CompressionStrategy strategy,
                                                                            SmooshedFileMapper fileMapper)

getLongSerializer

public static LongSupplierSerializer getLongSerializer(IOPeon ioPeon,
                                                       String filenameBase,
                                                       ByteOrder order,
                                                       CompressionFactory.LongEncodingStrategy encodingStrategy,
                                                       CompressedObjectStrategy.CompressionStrategy compressionStrategy)

getFloatSupplier

public static com.google.common.base.Supplier<IndexedFloats> getFloatSupplier(int totalSize,
                                                                              int sizePer,
                                                                              ByteBuffer fromBuffer,
                                                                              ByteOrder order,
                                                                              CompressedObjectStrategy.CompressionStrategy strategy,
                                                                              SmooshedFileMapper fileMapper)

getFloatSerializer

public static FloatSupplierSerializer getFloatSerializer(IOPeon ioPeon,
                                                         String filenameBase,
                                                         ByteOrder order,
                                                         CompressedObjectStrategy.CompressionStrategy compressionStrategy)