io.druid.segment

Class StringDimensionIndexer

java.lang.Object

io.druid.segment.StringDimensionIndexer

All Implemented Interfaces:

DimensionIndexer<Integer,int[],String>

public class StringDimensionIndexer
extends Object
implements DimensionIndexer<Integer,int[],String>

Field Summary

Fields

Modifier and Type	Field and Description
static com.google.common.base.Function<Object,String>	STRING_TRANSFORMER
static Comparator<String>	UNENCODED_COMPARATOR

Fields inherited from interface io.druid.segment.DimensionIndexer

ARRAY, LIST

Constructor Summary

Constructors

Constructor and Description
StringDimensionIndexer(DimensionSchema.MultiValueHandling multiValueHandling)

Method Summary

Modifier and Type	Method and Description
boolean	checkUnsortedEncodedKeyComponentsEqual(int[] lhs, int[] rhs) Check if two row value arrays from TimeAndDims keys are equal.
int	compareUnsortedEncodedKeyComponents(int[] lhs, int[] rhs) Compares the row values for this DimensionIndexer's dimension from a TimeAndDims key.
Object	convertUnsortedEncodedKeyComponentToActualArrayOrList(int[] key, boolean asList) Given a row value array from a TimeAndDims key, as described in the documentation for compareUnsortedEncodedKeyComponents(), convert the unsorted encoded values to a list or array of actual values.
int[]	convertUnsortedEncodedKeyComponentToSortedEncodedKeyComponent(int[] key) Given a row value array from a TimeAndDims key, as described in the documentation for compareUnsortedEncodedKeyComponents(), convert the unsorted encoded values to an array of sorted encoded values (i.e., sorted by their corresponding actual values)
void	fillBitmapsFromUnsortedEncodedKeyComponent(int[] key, int rowNum, MutableBitmap[] bitmapIndexes, BitmapFactory factory) Helper function for building bitmap indexes for integer-encoded dimensions.
int	getCardinality() Get the cardinality of this dimension's values.
String	getMaxValue() Get the maximum dimension value seen by this indexer.
String	getMinValue() Get the minimum dimension value seen by this indexer.
Integer	getSortedEncodedValueFromUnsorted(Integer unsortedIntermediateValue) Given an encoded value that was ordered by time of ingestion, return the equivalent encoded value ordered by associated actual value.
Indexed<String>	getSortedIndexedValues() Returns an indexed structure of this dimension's sorted actual values.
int	getUnsortedEncodedKeyComponentHashCode(int[] key) Given a row value array from a TimeAndDims key, generate a hashcode.
Integer	getUnsortedEncodedValueFromSorted(Integer sortedIntermediateValue) Given an encoded value that was ordered by associated actual value, return the equivalent encoded value ordered by time of ingestion.
ValueType	getValueType()
DimensionSelector	makeDimensionSelector(DimensionSpec spec, IncrementalIndexStorageAdapter.EntryHolder currEntry, IncrementalIndex.DimensionDesc desc) Return an object used to read values from this indexer's column as Strings.
FloatColumnSelector	makeFloatColumnSelector(IncrementalIndexStorageAdapter.EntryHolder currEntry, IncrementalIndex.DimensionDesc desc) Return an object used to read values from this indexer's column as Floats.
LongColumnSelector	makeLongColumnSelector(IncrementalIndexStorageAdapter.EntryHolder currEntry, IncrementalIndex.DimensionDesc desc) Return an object used to read values from this indexer's column as Longs.
ObjectColumnSelector	makeObjectColumnSelector(DimensionSpec spec, IncrementalIndexStorageAdapter.EntryHolder currEntry, IncrementalIndex.DimensionDesc desc) Return an object used to read values from this indexer's column as Objects.
int[]	processRowValsToUnsortedEncodedKeyComponent(Object dimValues) Given a single row value or list of row values (for multi-valued dimensions), update any internal data structures with the ingested values and return the row values as an array to be used within a TimeAndDims key.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STRING_TRANSFORMER

public static final com.google.common.base.Function<Object,String> STRING_TRANSFORMER

UNENCODED_COMPARATOR

public static final Comparator<String> UNENCODED_COMPARATOR

Constructor Detail

StringDimensionIndexer

public StringDimensionIndexer(DimensionSchema.MultiValueHandling multiValueHandling)

Method Detail

getValueType

public ValueType getValueType()

Specified by:

getValueType in interface DimensionIndexer<Integer,int[],String>

Returns:

The ValueType corresponding to this dimension indexer's ActualType.

processRowValsToUnsortedEncodedKeyComponent

public int[] processRowValsToUnsortedEncodedKeyComponent(Object dimValues)

Description copied from interface: DimensionIndexer

Given a single row value or list of row values (for multi-valued dimensions), update any internal data structures with the ingested values and return the row values as an array to be used within a TimeAndDims key. For example, the dictionary-encoded String-type column will return an int[] containing a dictionary ID. The value within the returned array should be encoded if applicable, i.e. as instances of EncodedType. NOTE: This function can change the internal state of the DimensionIndexer.

Specified by:

processRowValsToUnsortedEncodedKeyComponent in interface DimensionIndexer<Integer,int[],String>

Parameters:

dimValues - Single row val to process

Returns:

An array containing an encoded representation of the input row value.

getSortedEncodedValueFromUnsorted

public Integer getSortedEncodedValueFromUnsorted(Integer unsortedIntermediateValue)

Description copied from interface: DimensionIndexer

Given an encoded value that was ordered by time of ingestion, return the equivalent encoded value ordered by associated actual value. Using the example in the class description: getSortedEncodedValueFromUnsorted(0) would return 2

Specified by:

getSortedEncodedValueFromUnsorted in interface DimensionIndexer<Integer,int[],String>

Parameters:

unsortedIntermediateValue - value to convert

Returns:

converted value

getUnsortedEncodedValueFromSorted

public Integer getUnsortedEncodedValueFromSorted(Integer sortedIntermediateValue)

Description copied from interface: DimensionIndexer

Given an encoded value that was ordered by associated actual value, return the equivalent encoded value ordered by time of ingestion. Using the example in the class description: getUnsortedEncodedValueFromSorted(2) would return 0

Specified by:

getUnsortedEncodedValueFromSorted in interface DimensionIndexer<Integer,int[],String>

Parameters:

sortedIntermediateValue - value to convert

Returns:

converted value

getSortedIndexedValues

public Indexed<String> getSortedIndexedValues()

Description copied from interface: DimensionIndexer

Returns an indexed structure of this dimension's sorted actual values. The integer IDs represent the ordering of the sorted values. Using the example in the class description: "Apple"=0, "Hello"=1, "World"=2

Specified by:

getSortedIndexedValues in interface DimensionIndexer<Integer,int[],String>

Returns:

Sorted index of actual values

getMinValue

public String getMinValue()

Description copied from interface: DimensionIndexer

Get the minimum dimension value seen by this indexer. NOTE: On an in-memory segment (IncrementalIndex), we can determine min/max values by looking at the stream of row values seen in calls to processSingleRowValToIndexKey(). However, on a disk-backed segment (QueryableIndex), the numeric dimensions do not currently have any supporting index structures that can be used to efficiently determine min/max values. When numeric dimension support is added, the segment format should be changed to store min/max values, to avoid performing a full-column scan to determine these values for numeric dims.

Specified by:

getMinValue in interface DimensionIndexer<Integer,int[],String>

Returns:

min value

getMaxValue

public String getMaxValue()

Description copied from interface: DimensionIndexer

Get the maximum dimension value seen by this indexer.

Specified by:

getMaxValue in interface DimensionIndexer<Integer,int[],String>

Returns:

max value

getCardinality

public int getCardinality()

Description copied from interface: DimensionIndexer

Get the cardinality of this dimension's values.

Specified by:

getCardinality in interface DimensionIndexer<Integer,int[],String>

Returns:

value cardinality

compareUnsortedEncodedKeyComponents

public int compareUnsortedEncodedKeyComponents(int[] lhs,
                                               int[] rhs)

Description copied from interface: DimensionIndexer

Compares the row values for this DimensionIndexer's dimension from a TimeAndDims key. The dimension value arrays within a TimeAndDims key always use the "unsorted" ordering for encoded values. The row values are passed to this function as an Object, the implementer should cast them to the type appropriate for this dimension. For example, a dictionary encoded String implementation would cast the Objects as int[] arrays. When comparing, if the two arrays have different lengths, the shorter array should be ordered first. Otherwise, the implementer of this function should iterate through the unsorted encoded values, converting them to their actual type (e.g., performing a dictionary lookup for a dict-encoded String dimension), and comparing the actual values until a difference is found. Refer to StringDimensionIndexer.compareUnsortedEncodedKeyComponents() for a reference implementation.

Specified by:

compareUnsortedEncodedKeyComponents in interface DimensionIndexer<Integer,int[],String>

Parameters:

lhs - dimension value array from a TimeAndDims key

rhs - dimension value array from a TimeAndDims key

Returns:

comparison of the two arrays

checkUnsortedEncodedKeyComponentsEqual

public boolean checkUnsortedEncodedKeyComponentsEqual(int[] lhs,
                                                      int[] rhs)

Description copied from interface: DimensionIndexer

Check if two row value arrays from TimeAndDims keys are equal.

Specified by:

checkUnsortedEncodedKeyComponentsEqual in interface DimensionIndexer<Integer,int[],String>

Parameters:

lhs - dimension value array from a TimeAndDims key

rhs - dimension value array from a TimeAndDims key

Returns:

true if the two arrays are equal

getUnsortedEncodedKeyComponentHashCode

public int getUnsortedEncodedKeyComponentHashCode(int[] key)

Description copied from interface: DimensionIndexer

Given a row value array from a TimeAndDims key, generate a hashcode.

Specified by:

getUnsortedEncodedKeyComponentHashCode in interface DimensionIndexer<Integer,int[],String>

Parameters:

key - dimension value array from a TimeAndDims key

Returns:

hashcode of the array

makeDimensionSelector

public DimensionSelector makeDimensionSelector(DimensionSpec spec,
                                               IncrementalIndexStorageAdapter.EntryHolder currEntry,
                                               IncrementalIndex.DimensionDesc desc)

Description copied from interface: DimensionIndexer

Return an object used to read values from this indexer's column as Strings.

Specified by:

makeDimensionSelector in interface DimensionIndexer<Integer,int[],String>

Parameters:

spec - Specifies the output name of a dimension and any extraction functions to be applied.

currEntry - Provides access to the current TimeAndDims object in the Cursor

desc - Descriptor object for this dimension within an IncrementalIndex

Returns:

A new object that reads rows from currEntry

makeLongColumnSelector

public LongColumnSelector makeLongColumnSelector(IncrementalIndexStorageAdapter.EntryHolder currEntry,
                                                 IncrementalIndex.DimensionDesc desc)

Description copied from interface: DimensionIndexer

Return an object used to read values from this indexer's column as Longs.

Specified by:

makeLongColumnSelector in interface DimensionIndexer<Integer,int[],String>

Parameters:

currEntry - Provides access to the current TimeAndDims object in the Cursor

desc - Descriptor object for this dimension within an IncrementalIndex

Returns:

A new object that reads rows from currEntry

makeFloatColumnSelector

public FloatColumnSelector makeFloatColumnSelector(IncrementalIndexStorageAdapter.EntryHolder currEntry,
                                                   IncrementalIndex.DimensionDesc desc)

Description copied from interface: DimensionIndexer

Return an object used to read values from this indexer's column as Floats.

Specified by:

makeFloatColumnSelector in interface DimensionIndexer<Integer,int[],String>

Parameters:

currEntry - Provides access to the current TimeAndDims object in the Cursor

desc - Descriptor object for this dimension within an IncrementalIndex

Returns:

A new object that reads rows from currEntry

makeObjectColumnSelector

public ObjectColumnSelector makeObjectColumnSelector(DimensionSpec spec,
                                                     IncrementalIndexStorageAdapter.EntryHolder currEntry,
                                                     IncrementalIndex.DimensionDesc desc)

Description copied from interface: DimensionIndexer

Return an object used to read values from this indexer's column as Objects.

Specified by:

makeObjectColumnSelector in interface DimensionIndexer<Integer,int[],String>

Parameters:

spec - Specifies the output name of a dimension and any extraction functions to be applied.

currEntry - Provides access to the current TimeAndDims object in the Cursor

desc - Descriptor object for this dimension within an IncrementalIndex

Returns:

A new object that reads rows from currEntry

convertUnsortedEncodedKeyComponentToActualArrayOrList

public Object convertUnsortedEncodedKeyComponentToActualArrayOrList(int[] key,
                                                                    boolean asList)

Description copied from interface: DimensionIndexer

Given a row value array from a TimeAndDims key, as described in the documentation for compareUnsortedEncodedKeyComponents(), convert the unsorted encoded values to a list or array of actual values. If the key has one element, this method should return a single Object instead of an array or list, ignoring the asList parameter.

Specified by:

convertUnsortedEncodedKeyComponentToActualArrayOrList in interface DimensionIndexer<Integer,int[],String>

Parameters:

key - dimension value array from a TimeAndDims key

asList - if true, return an array; if false, return a list

Returns:

single value, array, or list containing the actual values corresponding to the encoded values in the input array

convertUnsortedEncodedKeyComponentToSortedEncodedKeyComponent

public int[] convertUnsortedEncodedKeyComponentToSortedEncodedKeyComponent(int[] key)

Description copied from interface: DimensionIndexer

Given a row value array from a TimeAndDims key, as described in the documentation for compareUnsortedEncodedKeyComponents(), convert the unsorted encoded values to an array of sorted encoded values (i.e., sorted by their corresponding actual values)

Specified by:

convertUnsortedEncodedKeyComponentToSortedEncodedKeyComponent in interface DimensionIndexer<Integer,int[],String>

Parameters:

key - dimension value array from a TimeAndDims key

Returns:

array containing the sorted encoded values corresponding to the unsorted encoded values in the input array

fillBitmapsFromUnsortedEncodedKeyComponent

public void fillBitmapsFromUnsortedEncodedKeyComponent(int[] key,
                                                       int rowNum,
                                                       MutableBitmap[] bitmapIndexes,
                                                       BitmapFactory factory)

Description copied from interface: DimensionIndexer

Helper function for building bitmap indexes for integer-encoded dimensions. Called by IncrementalIndexAdapter as it iterates through its sequence of rows. Given a row value array from a TimeAndDims key, with the current row number indicated by "rowNum", set the index for "rowNum" in the bitmap index for each value that appears in the row value array. For example, if key is an int[] array with values [1,3,4] for a dictionary-encoded String dimension, and rowNum is 27, this function would set bit 27 in bitmapIndexes[1], bitmapIndexes[3], and bitmapIndexes[4] See StringDimensionIndexer.fillBitmapsFromUnsortedEncodedKeyComponent() for a reference implementation. If a dimension type does not support bitmap indexes, this function will not be called and can be left unimplemented.

Specified by:

fillBitmapsFromUnsortedEncodedKeyComponent in interface DimensionIndexer<Integer,int[],String>

Parameters:

key - dimension value array from a TimeAndDims key

rowNum - current row number

bitmapIndexes - array of bitmaps, indexed by integer dimension value

factory - bitmap factory