io.druid.segment

Interface DimensionHandler<EncodedType extends Comparable<EncodedType>,EncodedKeyComponentType,ActualType extends Comparable<ActualType>>

Type Parameters:

EncodedType - class of a single encoded value

EncodedKeyComponentType - A row key contains a component for each dimension, this param specifies the class of this dimension's key component. A column type that supports multivalue rows should use an array type (Strings would use int[]). Column types without multivalue row support should use single objects (e.g., Long, Float).

ActualType - class of a single actual value

All Known Implementing Classes:

FloatDimensionHandler, LongDimensionHandler, StringDimensionHandler

public interface DimensionHandler<EncodedType extends Comparable<EncodedType>,EncodedKeyComponentType,ActualType extends Comparable<ActualType>>

Processing related interface A DimensionHandler is an object that encapsulates indexing, column merging/building, and querying operations for a given dimension type (e.g., dict-encoded String, Long). These operations are handled by sub-objects created through a DimensionHandler's methods: DimensionIndexer, DimensionMerger, and DimensionColumnReader, respectively. Each DimensionHandler object is associated with a single dimension. This interface allows type-specific behavior column logic, such as choice of indexing structures and disk formats. to be contained within a type-specific set of handler objects, simplifying processing classes such as IncrementalIndex and IndexMerger and allowing for abstracted development of additional dimension types. A dimension may have two representations, an encoded representation and a actual representation. For example, a value for a String dimension has an integer dictionary encoding, and an actual String representation. A DimensionHandler is a stateless object, and thus thread-safe; its methods should be pure functions. The EncodedType and ActualType are Comparable because columns used as dimensions must have sortable values.

Method Summary

Modifier and Type	Method and Description
int	compareSortedEncodedKeyComponents(EncodedKeyComponentType lhs, EncodedKeyComponentType rhs) Given two key components representing sorted encoded row value(s), return the result of their comparison.
String	getDimensionName() Get the name of the column associated with this handler.
EncodedKeyComponentType	getEncodedKeyComponentFromColumn(Closeable column, int currRow) Given a subcolumn from getSubColumn, and the index of the current row, retrieve a dimension's values from a row as an EncodedKeyComponentType.
int	getLengthOfEncodedKeyComponent(EncodedKeyComponentType dimVals) Given an key component representing a single set of row value(s) for this dimension as an Object, return the length of the key component after appropriate type-casting.
Closeable	getSubColumn(Column column) Given a Column, return a type-specific object that can be used to retrieve row values.
DimensionIndexer<EncodedType,EncodedKeyComponentType,ActualType>	makeIndexer() Creates a new DimensionIndexer, a per-dimension object responsible for processing ingested rows in-memory, used by the IncrementalIndex.
DimensionMergerLegacy<EncodedKeyComponentType>	makeLegacyMerger(IndexSpec indexSpec, File outDir, IOPeon ioPeon, ColumnCapabilities capabilities, ProgressIndicator progress) Creates a new DimensionMergerLegacy, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension.
DimensionMergerV9<EncodedKeyComponentType>	makeMerger(IndexSpec indexSpec, File outDir, IOPeon ioPeon, ColumnCapabilities capabilities, ProgressIndicator progress) Creates a new DimensionMergerV9, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension.
void	validateSortedEncodedKeyComponents(EncodedKeyComponentType lhs, EncodedKeyComponentType rhs, Indexed<ActualType> lhsEncodings, Indexed<ActualType> rhsEncodings) Given two key components representing sorted encoded row value(s), check that the two key components have the same encoded values, or if the encoded values differ, that they translate into the same actual values, using the mappings provided by lhsEncodings and rhsEncodings (if applicable).

Method Detail

getDimensionName

String getDimensionName()

Get the name of the column associated with this handler. This string would be the output name of the column during ingestion, and the name of an input column when querying.

Returns:

Dimension name

makeIndexer

DimensionIndexer<EncodedType,EncodedKeyComponentType,ActualType> makeIndexer()

Creates a new DimensionIndexer, a per-dimension object responsible for processing ingested rows in-memory, used by the IncrementalIndex. See DimensionIndexer interface for more information.

Returns:

A new DimensionIndexer object.

makeMerger

DimensionMergerV9<EncodedKeyComponentType> makeMerger(IndexSpec indexSpec,
                                                      File outDir,
                                                      IOPeon ioPeon,
                                                      ColumnCapabilities capabilities,
                                                      ProgressIndicator progress)
                                               throws IOException

Creates a new DimensionMergerV9, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension. For use with IndexMergerV9 only. See DimensionMergerV9 interface for more information.

Parameters:

indexSpec - Specification object for the index merge

outDir - Location to store files generated by the merging process

ioPeon - ioPeon object passed in by IndexMerger, manages files created by the merging process

capabilities - The ColumnCapabilities of the dimension represented by this DimensionHandler

progress - ProgressIndicator used by the merging process

Returns:

A new DimensionMergerV9 object.

Throws:

IOException

makeLegacyMerger

DimensionMergerLegacy<EncodedKeyComponentType> makeLegacyMerger(IndexSpec indexSpec,
                                                                File outDir,
                                                                IOPeon ioPeon,
                                                                ColumnCapabilities capabilities,
                                                                ProgressIndicator progress)
                                                         throws IOException

Creates a new DimensionMergerLegacy, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension. For use with IndexMerger only. See DimensionMergerLegacy interface for more information.

Parameters:

indexSpec - Specification object for the index merge

outDir - Location to store files generated by the merging process

ioPeon - ioPeon object passed in by IndexMerger, manages files created by the merging process

capabilities - The ColumnCapabilities of the dimension represented by this DimensionHandler

progress - ProgressIndicator used by the merging process

Returns:

A new DimensionMergerLegacy object.

Throws:

IOException

getLengthOfEncodedKeyComponent

int getLengthOfEncodedKeyComponent(EncodedKeyComponentType dimVals)

Given an key component representing a single set of row value(s) for this dimension as an Object, return the length of the key component after appropriate type-casting. For example, a dictionary encoded String dimension would receive an int[] as input to this method, while a Long numeric dimension would receive a single Long object (no multivalue support)

Parameters:

dimVals - Values for this dimension from a row

Returns:

Size of dimVals

compareSortedEncodedKeyComponents

int compareSortedEncodedKeyComponents(EncodedKeyComponentType lhs,
                                      EncodedKeyComponentType rhs)

Given two key components representing sorted encoded row value(s), return the result of their comparison. If the two key components have different lengths, the shorter component should be ordered first in the comparison. Otherwise, this function should iterate through the key components and return the comparison of the first difference. For dimensions that do not support multivalue rows, lhs and rhs can be compared directly.

Parameters:

lhs - key component from a row

rhs - key component from a row

Returns:

integer indicating comparison result of key components

validateSortedEncodedKeyComponents

void validateSortedEncodedKeyComponents(EncodedKeyComponentType lhs,
                                        EncodedKeyComponentType rhs,
                                        Indexed<ActualType> lhsEncodings,
                                        Indexed<ActualType> rhsEncodings)
                                 throws SegmentValidationException

Given two key components representing sorted encoded row value(s), check that the two key components have the same encoded values, or if the encoded values differ, that they translate into the same actual values, using the mappings provided by lhsEncodings and rhsEncodings (if applicable). If validation fails, this method should throw a SegmentValidationException. Used by IndexIO for validating segments. See StringDimensionHandler.validateSortedEncodedKeyComponents() for a reference implementation.

Parameters:

lhs - key component from a row

rhs - key component from a row

lhsEncodings - encoding lookup from lhs's segment, null if not applicable for this dimension's type

rhsEncodings - encoding lookup from rhs's segment, null if not applicable for this dimension's type

Throws:

SegmentValidationException

getSubColumn

Closeable getSubColumn(Column column)

Given a Column, return a type-specific object that can be used to retrieve row values. For example: - A String-typed implementation would return the result of column.getDictionaryEncoding() - A long-typed implemention would return the result of column.getGenericColumn().

Parameters:

column - Column for this dimension from a QueryableIndex

Returns:

The type-specific column subobject for this dimension.

getEncodedKeyComponentFromColumn

EncodedKeyComponentType getEncodedKeyComponentFromColumn(Closeable column,
                                                         int currRow)

Given a subcolumn from getSubColumn, and the index of the current row, retrieve a dimension's values from a row as an EncodedKeyComponentType. For example: - A String-typed implementation would read the current row from a DictionaryEncodedColumn as an int[]. - A long-typed implemention would read the current row from a GenericColumn and return a Long.

Parameters:

column - Column for this dimension from a QueryableIndex

currRow - The index of the row to retrieve

Returns:

The key component for this dimension from the current row of the column.