io.druid.segment

Class StringDimensionMergerV9

java.lang.Object

io.druid.segment.StringDimensionMergerV9

All Implemented Interfaces:

DimensionMerger<int[]>, DimensionMergerV9<int[]>

public class StringDimensionMergerV9
extends Object
implements DimensionMergerV9<int[]>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StringDimensionMergerV9.ConvertingBitmapValues
protected static interface	StringDimensionMergerV9.IndexSeeker
protected static class	StringDimensionMergerV9.IndexSeekerWithConversion Get old dictId from new dictId, and only support access in order
protected static class	StringDimensionMergerV9.IndexSeekerWithoutConversion

Constructor Summary

Constructors

Constructor and Description
StringDimensionMergerV9(String dimensionName, IndexSpec indexSpec, SegmentWriteOutMedium segmentWriteOutMedium, ColumnCapabilities capabilities, ProgressIndicator progress)

Method Summary

Modifier and Type	Method and Description
boolean	canSkip() Return true if this dimension's data does not need to be written to the segment.
int[]	convertSegmentRowValuesToMergedRowValues(int[] segmentRow, int segmentIndexNumber) Convert a row's key component with per-segment encoding to its equivalent representation in the merged set of rows.
protected boolean	isNullColumn(Iterable<String> dimValues)
ColumnDescriptor	makeColumnDescriptor() Return a ColumnDescriptor containing ColumnPartSerde objects appropriate for this DimensionMerger's value metadata, sequence of row values, and index structures.
void	processMergedRow(int[] rowValues) Process a key component from the merged sequence of rows and update the DimensionMerger's internal state.
protected void	processMergedRowHelper(int[] vals)
protected void	setupEncodedValueWriter()
protected StringDimensionMergerV9.IndexSeeker[]	toIndexSeekers(List<IndexableAdapter> adapters, ArrayList<IntBuffer> dimConversions, String dimension)
void	writeIndexes(List<IntBuffer> segmentRowNumConversions) Internally construct any index structures relevant to this DimensionMerger.
void	writeMergedValueMetadata(List<IndexableAdapter> adapters) Given a list of segment adapters: - Read any value metadata (e.g., dictionary encoding information) from the adapters - Merge this value metadata and update the internal state of the implementing class.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StringDimensionMergerV9

public StringDimensionMergerV9(String dimensionName,
                               IndexSpec indexSpec,
                               SegmentWriteOutMedium segmentWriteOutMedium,
                               ColumnCapabilities capabilities,
                               ProgressIndicator progress)

Method Detail

writeMergedValueMetadata

public void writeMergedValueMetadata(List<IndexableAdapter> adapters)
                              throws IOException

Description copied from interface: DimensionMerger

Given a list of segment adapters: - Read any value metadata (e.g., dictionary encoding information) from the adapters - Merge this value metadata and update the internal state of the implementing class. The implementer should maintain knowledge of the "index number" of the adapters in the input list, i.e., the position of each adapter in the input list. This "index number" will be used to refer to specific segments later in convertSegmentRowValuesToMergedRowValues(). Otherwise, the details of how this merging occurs and how to store the merged data is left to the implementer.

Specified by:

writeMergedValueMetadata in interface DimensionMerger<int[]>

Parameters:

adapters - List of adapters to be merged.

Throws:

IOException

setupEncodedValueWriter

protected void setupEncodedValueWriter()
                                throws IOException

Throws:

IOException

convertSegmentRowValuesToMergedRowValues

public int[] convertSegmentRowValuesToMergedRowValues(int[] segmentRow,
                                                      int segmentIndexNumber)

Description copied from interface: DimensionMerger

Convert a row's key component with per-segment encoding to its equivalent representation in the merged set of rows. This function is used by the index merging process to build the merged sequence of rows. The implementing class is expected to use the merged value metadata constructed during writeMergedValueMetadata, if applicable. For example, an implementation of this function for a dictionary-encoded String column would convert the segment-specific dictionary values within the row to the common merged dictionary values determined during writeMergedValueMetadata().

Specified by:

convertSegmentRowValuesToMergedRowValues in interface DimensionMerger<int[]>

Parameters:

segmentRow - A row's key component for this dimension. The encoding of the key component's values will be converted from per-segment encodings to the combined encodings from the merged sequence of rows.

segmentIndexNumber - Integer indicating which segment the row originated from.

processMergedRow

public void processMergedRow(int[] rowValues)
                      throws IOException

Description copied from interface: DimensionMerger

Process a key component from the merged sequence of rows and update the DimensionMerger's internal state. After constructing a merged sequence of rows across segments, the index merging process will iterate through these rows and pass row key components from each dimension to their corresponding DimensionMergers. This allows each DimensionMerger to build its internal view of the sequence of merged rows, to be written out to a segment later.

Specified by:

processMergedRow in interface DimensionMerger<int[]>

Parameters:

rowValues - The row values to be added.

Throws:

IOException

processMergedRowHelper

protected void processMergedRowHelper(int[] vals)
                               throws IOException

Throws:

IOException

writeIndexes

public void writeIndexes(List<IntBuffer> segmentRowNumConversions)
                  throws IOException

Description copied from interface: DimensionMerger

Internally construct any index structures relevant to this DimensionMerger. After receiving the sequence of merged rows via iterated processMergedRow() calls, the DimensionMerger can now build any index structures it needs. For example, a dictionary encoded String implementation would create its bitmap indexes for the merged segment during this step. The index merger will provide a list of row number conversion IntBuffer objects. Each IntBuffer is associated with one of the segments being merged; the position of the IntBuffer in the list corresponds to the position of segment adapters within the input list of writeMergedValueMetadata(). For example, suppose there are two segments A and B. Row 24 from segment A maps to row 99 in the merged sequence of rows, The IntBuffer for segment A would have a mapping of 24 -> 99.

Specified by:

writeIndexes in interface DimensionMerger<int[]>

Parameters:

segmentRowNumConversions - A list of row number conversion IntBuffer objects.

Throws:

IOException

canSkip

public boolean canSkip()

Description copied from interface: DimensionMerger

Return true if this dimension's data does not need to be written to the segment. For example, if a dictionary-encoded String dimension had only null values, it can be skipped.

Specified by:

canSkip in interface DimensionMerger<int[]>

Returns:

true if this dimension can be excluded from the merged segment.

makeColumnDescriptor

public ColumnDescriptor makeColumnDescriptor()

Description copied from interface: DimensionMergerV9

Return a ColumnDescriptor containing ColumnPartSerde objects appropriate for this DimensionMerger's value metadata, sequence of row values, and index structures.

Specified by:

makeColumnDescriptor in interface DimensionMergerV9<int[]>

Returns:

ColumnDescriptor that IndexMergerV9 will use to build a column.

toIndexSeekers

protected StringDimensionMergerV9.IndexSeeker[] toIndexSeekers(List<IndexableAdapter> adapters,
                                                               ArrayList<IntBuffer> dimConversions,
                                                               String dimension)

isNullColumn

protected boolean isNullColumn(Iterable<String> dimValues)