public class CompressionFactory extends Object
CompressionStrategy
and Encoding(such as CompressionFactory.LongEncodingStrategy
for type Long). CompressionStrategy is unaware of the data type
and is based on byte operations. It must compress and decompress in block of bytes. Encoding refers to compression
method relies on data format, so a different set of Encodings exist for each data type.
Storage Format : Byte 1 : version (currently 0x02) Byte 2 - 5 : number of values Byte 6 - 9 : size per block (even if block format isn't used, this is needed for backward compatibility) Byte 10 : compression strategy (contains a flag if there's an encoding byte, see below for how the flag is defined) Byte 11(optional) : encoding type
Encoding specific header (described below)
Block related header (if block compression is used, described in GenericIndexed)
Values
Modifier and Type | Class and Description |
---|---|
static class |
CompressionFactory.LongEncodingFormat |
static interface |
CompressionFactory.LongEncodingReader |
static class |
CompressionFactory.LongEncodingStrategy
The compression of decompression of encodings are separated into different enums.
|
static interface |
CompressionFactory.LongEncodingWriter
This writer output encoded values to the given ByteBuffer or OutputStream.
|
Modifier and Type | Field and Description |
---|---|
static CompressionFactory.LongEncodingStrategy |
DEFAULT_LONG_ENCODING_STRATEGY |
static byte |
DELTA_ENCODING_VERSION
Delta Encoding Header v1:
Byte 1 : version
Byte 2 - 9 : base value
Byte 10 - 13 : number of bits per value
|
static CompressionFactory.LongEncodingFormat |
LEGACY_LONG_ENCODING_FORMAT |
static int |
MAX_TABLE_SIZE |
static byte |
TABLE_ENCODING_VERSION
Table Encoding Header v1 :
Byte 1 : version
Byte 2 - 5 : table size
Byte 6 - (6 + 8 * table size - 1) : table of encoding, where the ith 8-byte value is encoded as i
|
Modifier and Type | Method and Description |
---|---|
static byte |
clearEncodingFlag(byte strategyId) |
static ColumnarDoublesSerializer |
getDoubleSerializer(SegmentWriteOutMedium segmentWriteOutMedium,
String filenameBase,
ByteOrder byteOrder,
CompressionStrategy compression) |
static com.google.common.base.Supplier<ColumnarDoubles> |
getDoubleSupplier(int totalSize,
int sizePer,
ByteBuffer fromBuffer,
ByteOrder byteOrder,
CompressionStrategy strategy) |
static ColumnarFloatsSerializer |
getFloatSerializer(SegmentWriteOutMedium segmentWriteOutMedium,
String filenameBase,
ByteOrder order,
CompressionStrategy compressionStrategy) |
static com.google.common.base.Supplier<ColumnarFloats> |
getFloatSupplier(int totalSize,
int sizePer,
ByteBuffer fromBuffer,
ByteOrder order,
CompressionStrategy strategy) |
static ColumnarLongsSerializer |
getLongSerializer(SegmentWriteOutMedium segmentWriteOutMedium,
String filenameBase,
ByteOrder order,
CompressionFactory.LongEncodingStrategy encodingStrategy,
CompressionStrategy compressionStrategy) |
static com.google.common.base.Supplier<ColumnarLongs> |
getLongSupplier(int totalSize,
int sizePer,
ByteBuffer fromBuffer,
ByteOrder order,
CompressionFactory.LongEncodingFormat encodingFormat,
CompressionStrategy strategy) |
static boolean |
hasEncodingFlag(byte strategyId) |
static byte |
setEncodingFlag(byte strategyId) |
public static final CompressionFactory.LongEncodingStrategy DEFAULT_LONG_ENCODING_STRATEGY
public static final CompressionFactory.LongEncodingFormat LEGACY_LONG_ENCODING_FORMAT
public static final byte DELTA_ENCODING_VERSION
public static final byte TABLE_ENCODING_VERSION
public static final int MAX_TABLE_SIZE
public static boolean hasEncodingFlag(byte strategyId)
public static byte setEncodingFlag(byte strategyId)
public static byte clearEncodingFlag(byte strategyId)
public static com.google.common.base.Supplier<ColumnarLongs> getLongSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionFactory.LongEncodingFormat encodingFormat, CompressionStrategy strategy)
public static ColumnarLongsSerializer getLongSerializer(SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionFactory.LongEncodingStrategy encodingStrategy, CompressionStrategy compressionStrategy)
public static com.google.common.base.Supplier<ColumnarFloats> getFloatSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder order, CompressionStrategy strategy)
public static ColumnarFloatsSerializer getFloatSerializer(SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder order, CompressionStrategy compressionStrategy)
public static com.google.common.base.Supplier<ColumnarDoubles> getDoubleSupplier(int totalSize, int sizePer, ByteBuffer fromBuffer, ByteOrder byteOrder, CompressionStrategy strategy)
public static ColumnarDoublesSerializer getDoubleSerializer(SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, ByteOrder byteOrder, CompressionStrategy compression)
Copyright © 2011–2018. All rights reserved.