public class ConcurrentGrouper<KeyType> extends Object implements Grouper<KeyType>
SpillingGrouper
instances. Thread-safe.
The passed-in buffer is cut up into concurrencyHint slices, and each slice is passed to a different underlying grouper. Access to each slice is separately synchronized. As long as the result set fits in memory, keys are partitioned between buffers based on their hash, and multiple threads can write into the same buffer. When it becomes clear that the result set does not fit in memory, the table switches to a mode where each thread gets its own buffer and its own spill files on disk.
Grouper.BufferComparator, Grouper.Entry<T>, Grouper.KeySerde<T>, Grouper.KeySerdeFactory<T>
Constructor and Description |
---|
ConcurrentGrouper(GroupByQueryConfig groupByQueryConfig,
com.google.common.base.Supplier<ByteBuffer> bufferSupplier,
com.google.common.base.Supplier<ResourceHolder<ByteBuffer>> combineBufferSupplier,
Grouper.KeySerdeFactory<KeyType> keySerdeFactory,
Grouper.KeySerdeFactory<KeyType> combineKeySerdeFactory,
ColumnSelectorFactory columnSelectorFactory,
AggregatorFactory[] aggregatorFactories,
LimitedTemporaryStorage temporaryStorage,
com.fasterxml.jackson.databind.ObjectMapper spillMapper,
int concurrencyHint,
DefaultLimitSpec limitSpec,
boolean sortHasNonGroupingFields,
com.google.common.util.concurrent.ListeningExecutorService executor,
int priority,
boolean hasQueryTimeout,
long queryTimeoutAt) |
Modifier and Type | Method and Description |
---|---|
AggregateResult |
aggregate(KeyType key,
int keyHash)
Aggregate the current row with the provided key.
|
void |
close()
Close the grouper and release associated resources.
|
void |
init()
Initialize the grouper.
|
boolean |
isInitialized()
Check this grouper is initialized or not.
|
CloseableIterator<Grouper.Entry<KeyType>> |
iterator(boolean sorted)
Iterate through entries.
|
void |
reset()
Reset the grouper to its initial state.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
aggregate, hashFunction
public ConcurrentGrouper(GroupByQueryConfig groupByQueryConfig, com.google.common.base.Supplier<ByteBuffer> bufferSupplier, com.google.common.base.Supplier<ResourceHolder<ByteBuffer>> combineBufferSupplier, Grouper.KeySerdeFactory<KeyType> keySerdeFactory, Grouper.KeySerdeFactory<KeyType> combineKeySerdeFactory, ColumnSelectorFactory columnSelectorFactory, AggregatorFactory[] aggregatorFactories, LimitedTemporaryStorage temporaryStorage, com.fasterxml.jackson.databind.ObjectMapper spillMapper, int concurrencyHint, DefaultLimitSpec limitSpec, boolean sortHasNonGroupingFields, com.google.common.util.concurrent.ListeningExecutorService executor, int priority, boolean hasQueryTimeout, long queryTimeoutAt)
public void init()
Grouper
Grouper.aggregate(Object)
and Grouper.aggregate(Object, int)
.public boolean isInitialized()
Grouper
isInitialized
in interface Grouper<KeyType>
public AggregateResult aggregate(KeyType key, int keyHash)
Grouper
aggregate
in interface Grouper<KeyType>
key
- key objectkeyHash
- result of Grouper.hashFunction()
on the keypublic void reset()
Grouper
public CloseableIterator<Grouper.Entry<KeyType>> iterator(boolean sorted)
Grouper
Some implementations allow writes even after this method is called. After you are done with the iterator
returned by this method, you should either call Grouper.close()
(if you are done with the Grouper) or
Grouper.reset()
(if you want to reuse it). Some implementations allow calling Grouper.iterator(boolean)
again if
you want another iterator. But, this method must not be called by multiple threads concurrently.
If "sorted" is true then the iterator will return sorted results. It will use KeyType's natural ordering on
deserialized objects, and will use the KeySerde#comparator()
on serialized objects. Woe be unto you
if these comparators are not equivalent.
Callers must process and discard the returned Grouper.Entry
s immediately because some implementations can reuse the
key objects.
Copyright © 2011–2018. All rights reserved.