io.druid.query.groupby.epinephelinae

Class StreamingMergeSortedGrouper<KeyType>

java.lang.Object

io.druid.query.groupby.epinephelinae.StreamingMergeSortedGrouper<KeyType>

All Implemented Interfaces:

Grouper<KeyType>, Closeable, AutoCloseable

public class StreamingMergeSortedGrouper<KeyType>
extends Object
implements Grouper<KeyType>

A streaming grouper which can aggregate sorted inputs. This grouper can aggregate while its iterator is being consumed. The aggregation thread and the iterating thread can be different. This grouper is backed by an off-heap circular array. The reading thread is able to read data from an array slot only when aggregation for the grouping key correspoing to that slot is finished. Since the reading and writing threads cannot access the same array slot at the same time, they can read/write data without contention. This class uses the spinlock for waiting for at least one slot to become available when the array is empty or full. If the array is empty, the reading thread waits for the aggregation for an array slot is finished. If the array is full, the writing thread waits for the reading thread to read at least one aggregate from the array.

Nested Class Summary

Nested classes/interfaces inherited from interface io.druid.query.groupby.epinephelinae.Grouper

Grouper.BufferComparator, Grouper.Entry<T>, Grouper.KeySerde<T>, Grouper.KeySerdeFactory<T>

Method Summary

Modifier and Type	Method and Description
AggregateResult	aggregate(KeyType key) Aggregate the current row with the provided key.
AggregateResult	aggregate(KeyType key, int notUsed) Aggregate the current row with the provided key.
void	close() Close the grouper and release associated resources.
void	finish() Signal that no more inputs are added.
void	init() Initialize the grouper.
boolean	isInitialized() Check this grouper is initialized or not.
CloseableIterator<Grouper.Entry<KeyType>>	iterator() Return a sorted iterator.
CloseableIterator<Grouper.Entry<KeyType>>	iterator(boolean sorted) Return a sorted iterator.
static <KeyType> int	requiredBufferCapacity(Grouper.KeySerde<KeyType> keySerde, AggregatorFactory[] aggregatorFactories) Returns the minimum buffer capacity required for this grouper.
void	reset() Reset the grouper to its initial state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.druid.query.groupby.epinephelinae.Grouper

hashFunction

Method Detail

requiredBufferCapacity

public static <KeyType> int requiredBufferCapacity(Grouper.KeySerde<KeyType> keySerde,
                                                   AggregatorFactory[] aggregatorFactories)

Returns the minimum buffer capacity required for this grouper. This grouper keeps track read/write indexes and they cannot point the same array slot at the same time. Since the read/write indexes move circularly, one extra slot is needed in addition to the read/write slots. Finally, the required minimum buffer capacity is 3 * record size.

Returns:

required minimum buffer capacity

init

public void init()

Description copied from interface: Grouper

Initialize the grouper. This method needs to be called before calling Grouper.aggregate(Object) and Grouper.aggregate(Object, int).

Specified by:

init in interface Grouper<KeyType>

isInitialized

public boolean isInitialized()

Description copied from interface: Grouper

Check this grouper is initialized or not.

Specified by:

isInitialized in interface Grouper<KeyType>

Returns:

true if the grouper is already initialized, otherwise false.

aggregate

public AggregateResult aggregate(KeyType key,
                                 int notUsed)

Description copied from interface: Grouper

Aggregate the current row with the provided key. Some implementations are thread-safe and some are not.

Specified by:

aggregate in interface Grouper<KeyType>

Parameters:

key - key object

notUsed - result of Grouper.hashFunction() on the key

Returns:

result that is ok if the row was aggregated, not ok if a resource limit was hit

aggregate

public AggregateResult aggregate(KeyType key)

Description copied from interface: Grouper

Aggregate the current row with the provided key. Some implementations are thread-safe and some are not.

Specified by:

aggregate in interface Grouper<KeyType>

Parameters:

key - key

Returns:

result that is ok if the row was aggregated, not ok if a resource limit was hit

reset

public void reset()

Description copied from interface: Grouper

Reset the grouper to its initial state.

Specified by:

reset in interface Grouper<KeyType>

close

public void close()

Description copied from interface: Grouper

Close the grouper and release associated resources.

Specified by:

close in interface Grouper<KeyType>

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

finish

public void finish()

Signal that no more inputs are added. Must be called after aggregate(Object) is called for the last input.

iterator

public CloseableIterator<Grouper.Entry<KeyType>> iterator()

Return a sorted iterator. This method can be called safely while writing, and the iterating thread and the writing thread can be different. The result iterator always returns sorted results. This method should be called only one time per grouper.

Returns:

a sorted iterator

iterator

public CloseableIterator<Grouper.Entry<KeyType>> iterator(boolean sorted)

Return a sorted iterator. This method can be called safely while writing and iterating thread and writing thread can be different. The result iterator always returns sorted results. This method should be called only one time per grouper.

Specified by:

iterator in interface Grouper<KeyType>

Parameters:

sorted - not used

Returns:

a sorted iterator