io.druid.segment

Class LongDimensionMergerV9

java.lang.Object

io.druid.segment.LongDimensionMergerV9

All Implemented Interfaces:

DimensionMerger<Long>, DimensionMergerV9<Long>

Direct Known Subclasses:

LongDimensionMergerLegacy

public class LongDimensionMergerV9
extends Object
implements DimensionMergerV9<Long>

Field Summary

Fields

Modifier and Type	Field and Description
protected ColumnCapabilities	capabilities
protected String	dimensionName
protected IndexSpec	indexSpec
protected IOPeon	ioPeon
protected File	outDir
protected ProgressIndicator	progress
protected LongColumnSerializer	serializer

Constructor Summary

Constructors

Constructor and Description
LongDimensionMergerV9(String dimensionName, IndexSpec indexSpec, File outDir, IOPeon ioPeon, ColumnCapabilities capabilities, ProgressIndicator progress)

Method Summary

Modifier and Type	Method and Description
boolean	canSkip() Return true if this dimension's data does not need to be written to the segment.
Long	convertSegmentRowValuesToMergedRowValues(Long segmentRow, int segmentIndexNumber) Convert a row's key component with per-segment encoding to its equivalent representation in the merged set of rows.
ColumnDescriptor	makeColumnDescriptor() Return a ColumnDescriptor containing ColumnPartSerde objects appropriate for this DimensionMerger's value metadata, sequence of row values, and index structures.
void	processMergedRow(Long rowValues) Process a key component from the merged sequence of rows and update the DimensionMerger's internal state.
protected void	setupEncodedValueWriter()
void	writeIndexes(List<IntBuffer> segmentRowNumConversions, com.google.common.io.Closer closer) Internally construct any index structures relevant to this DimensionMerger.
void	writeMergedValueMetadata(List<IndexableAdapter> adapters) Given a list of segment adapters: - Read any value metadata (e.g., dictionary encoding information) from the adapters - Merge this value metadata and update the internal state of the implementing class.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dimensionName

protected String dimensionName

progress

protected ProgressIndicator progress

indexSpec

protected final IndexSpec indexSpec

capabilities

protected ColumnCapabilities capabilities

outDir

protected final File outDir

ioPeon

protected IOPeon ioPeon

serializer

protected LongColumnSerializer serializer

Constructor Detail

LongDimensionMergerV9

public LongDimensionMergerV9(String dimensionName,
                             IndexSpec indexSpec,
                             File outDir,
                             IOPeon ioPeon,
                             ColumnCapabilities capabilities,
                             ProgressIndicator progress)

Method Detail

setupEncodedValueWriter

protected void setupEncodedValueWriter()
                                throws IOException

Throws:

IOException

writeMergedValueMetadata

public void writeMergedValueMetadata(List<IndexableAdapter> adapters)
                              throws IOException

Description copied from interface: DimensionMerger

Given a list of segment adapters: - Read any value metadata (e.g., dictionary encoding information) from the adapters - Merge this value metadata and update the internal state of the implementing class. The implementer should maintain knowledge of the "index number" of the adapters in the input list, i.e., the position of each adapter in the input list. This "index number" will be used to refer to specific segments later in convertSegmentRowValuesToMergedRowValues(). Otherwise, the details of how this merging occurs and how to store the merged data is left to the implementer.

Specified by:

writeMergedValueMetadata in interface DimensionMerger<Long>

Parameters:

adapters - List of adapters to be merged.

Throws:

IOException

convertSegmentRowValuesToMergedRowValues

public Long convertSegmentRowValuesToMergedRowValues(Long segmentRow,
                                                     int segmentIndexNumber)

Description copied from interface: DimensionMerger

Convert a row's key component with per-segment encoding to its equivalent representation in the merged set of rows. This function is used by the index merging process to build the merged sequence of rows. The implementing class is expected to use the merged value metadata constructed during writeMergedValueMetadata, if applicable. For example, an implementation of this function for a dictionary-encoded String column would convert the segment-specific dictionary values within the row to the common merged dictionary values determined during writeMergedValueMetadata().

Specified by:

convertSegmentRowValuesToMergedRowValues in interface DimensionMerger<Long>

Parameters:

segmentRow - A row's key component for this dimension. The encoding of the key component's values will be converted from per-segment encodings to the combined encodings from the merged sequence of rows.

segmentIndexNumber - Integer indicating which segment the row originated from.

processMergedRow

public void processMergedRow(Long rowValues)
                      throws IOException

Description copied from interface: DimensionMerger

Process a key component from the merged sequence of rows and update the DimensionMerger's internal state. After constructing a merged sequence of rows across segments, the index merging process will iterate through these rows and pass row key components from each dimension to their corresponding DimensionMergers. This allows each DimensionMerger to build its internal view of the sequence of merged rows, to be written out to a segment later.

Specified by:

processMergedRow in interface DimensionMerger<Long>

Parameters:

rowValues - The row values to be added.

Throws:

IOException

writeIndexes

public void writeIndexes(List<IntBuffer> segmentRowNumConversions,
                         com.google.common.io.Closer closer)
                  throws IOException

Description copied from interface: DimensionMerger

Internally construct any index structures relevant to this DimensionMerger. After receiving the sequence of merged rows via iterated processMergedRow() calls, the DimensionMerger can now build any index structures it needs. For example, a dictionary encoded String implementation would create its bitmap indexes for the merged segment during this step. The index merger will provide a list of row number conversion IntBuffer objects. Each IntBuffer is associated with one of the segments being merged; the position of the IntBuffer in the list corresponds to the position of segment adapters within the input list of writeMergedValueMetadata(). For example, suppose there are two segments A and B. Row 24 from segment A maps to row 99 in the merged sequence of rows, The IntBuffer for segment A would have a mapping of 24 -> 99.

Specified by:

writeIndexes in interface DimensionMerger<Long>

Parameters:

segmentRowNumConversions - A list of row number conversion IntBuffer objects.

closer - Add Closeables for resource cleanup to this Closer if needed

Throws:

IOException

canSkip

public boolean canSkip()

Description copied from interface: DimensionMerger

Return true if this dimension's data does not need to be written to the segment. For example, if a dictionary-encoded String dimension had only null values, it can be skipped.

Specified by:

canSkip in interface DimensionMerger<Long>

Returns:

true if this dimension can be excluded from the merged segment.

makeColumnDescriptor

public ColumnDescriptor makeColumnDescriptor()
                                      throws IOException

Description copied from interface: DimensionMergerV9

Return a ColumnDescriptor containing ColumnPartSerde objects appropriate for this DimensionMerger's value metadata, sequence of row values, and index structures.

Specified by:

makeColumnDescriptor in interface DimensionMergerV9<Long>

Returns:

ColumnDescriptor that IndexMergerV9 will use to build a column.

Throws:

IOException