io.druid.segment.data

Class GenericIndexed<T>

java.lang.Object

io.druid.segment.data.GenericIndexed<T>

All Implemented Interfaces:

Indexed<T>, Iterable<T>

public class GenericIndexed<T>
extends Object
implements Indexed<T>

A generic, flat storage mechanism. Use static methods fromArray() or fromIterable() to construct. If input is sorted, supports binary search index lookups. If input is not sorted, only supports array-like index lookups.

V1 Storage Format:

byte 1: version (0x1) byte 2 == 0x1 =>; allowReverseLookup bytes 3-6 =>; numBytesUsed bytes 7-10 =>; numElements bytes 10-((numElements * 4) + 10): integers representing *end* offsets of byte serialized values bytes ((numElements * 4) + 10)-(numBytesUsed + 2): 4-byte integer representing length of value, followed by bytes for value

V2 Storage Format Meta, header and value files are separate and header file stored in native endian byte order. Meta File: byte 1: version (0x2) byte 2 == 0x1 =>; allowReverseLookup bytes 3-6: numberOfElementsPerValueFile expressed as power of 2. That means all the value files contains same number of items except last value file and may have fewer elements. bytes 7-10 =>; numElements bytes 11-14 =>; columnNameLength bytes 15-columnNameLength =>; columnName

Header file name is identified as: String.format("%s_header", columnName) value files are identified as: String.format("%s_value_%d", columnName, fileNumber) number of value files == numElements/numberOfElementsPerValueFile

Field Summary

Fields

Modifier and Type	Field and Description
static ObjectStrategy<String>	STRING_STRATEGY
static byte	VERSION_ONE
static byte	VERSION_TWO

Method Summary

Modifier and Type	Method and Description
static <T> GenericIndexed<T>	fromArray(T[] objects, ObjectStrategy<T> strategy)
static <T> GenericIndexed<T>	fromIterable(Iterable<T> objectsIterable, ObjectStrategy<T> strategy)
T	get(int index)
Class<? extends T>	getClazz()
static int	getNumberOfFilesRequired(int bagSize, long numWritten)
long	getSerializedSize()
int	indexOf(T value) Returns the index of "value" in this GenericIndexed object, or (-(insertion point) - 1) if the value is not present, in the manner of Arrays.binarySearch.
Iterator<T>	iterator()
static <T> GenericIndexed<T>	read(ByteBuffer buffer, ObjectStrategy<T> strategy)
static <T> GenericIndexed<T>	read(ByteBuffer buffer, ObjectStrategy<T> strategy, SmooshedFileMapper fileMapper)
io.druid.segment.data.GenericIndexed.BufferIndexed	singleThreaded() Create a non-thread-safe Indexed, which may perform better than the underlying Indexed.
int	size()
void	writeToChannel(WritableByteChannel channel)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

VERSION_ONE

public static final byte VERSION_ONE

See Also:

Constant Field Values

VERSION_TWO

public static final byte VERSION_TWO

See Also:

Constant Field Values

STRING_STRATEGY

public static final ObjectStrategy<String> STRING_STRATEGY

Method Detail

getNumberOfFilesRequired

public static int getNumberOfFilesRequired(int bagSize,
                                           long numWritten)

fromArray

public static <T> GenericIndexed<T> fromArray(T[] objects,
                                              ObjectStrategy<T> strategy)

fromIterable

public static <T> GenericIndexed<T> fromIterable(Iterable<T> objectsIterable,
                                                 ObjectStrategy<T> strategy)

read

public static <T> GenericIndexed<T> read(ByteBuffer buffer,
                                         ObjectStrategy<T> strategy)

read

public static <T> GenericIndexed<T> read(ByteBuffer buffer,
                                         ObjectStrategy<T> strategy,
                                         SmooshedFileMapper fileMapper)

getClazz

public Class<? extends T> getClazz()

Specified by:

getClazz in interface Indexed<T>

size

public int size()

Specified by:

size in interface Indexed<T>

get

public T get(int index)

Specified by:

get in interface Indexed<T>

indexOf

public int indexOf(T value)

Returns the index of "value" in this GenericIndexed object, or (-(insertion point) - 1) if the value is not present, in the manner of Arrays.binarySearch. This strengthens the contract of Indexed, which only guarantees that values-not-found will return some negative number.

Specified by:

indexOf in interface Indexed<T>

Parameters:

value - value to search for

Returns:

index of value, or negative number equal to (-(insertion point) - 1).

iterator

public Iterator<T> iterator()

Specified by:

iterator in interface Iterable<T>

getSerializedSize

public long getSerializedSize()

writeToChannel

public void writeToChannel(WritableByteChannel channel)
                    throws IOException

Throws:

IOException

singleThreaded

public io.druid.segment.data.GenericIndexed.BufferIndexed singleThreaded()

Create a non-thread-safe Indexed, which may perform better than the underlying Indexed.

Returns:

a non-thread-safe Indexed