public class SpillingGrouper<KeyType> extends Object implements Grouper<KeyType>
BufferHashGrouper
. Not thread-safe.
When the underlying grouper is full, its contents are sorted and written to temporary files using "spillMapper".Grouper.BufferComparator, Grouper.Entry<T>, Grouper.KeySerde<T>, Grouper.KeySerdeFactory<T>
Constructor and Description |
---|
SpillingGrouper(com.google.common.base.Supplier<ByteBuffer> bufferSupplier,
Grouper.KeySerdeFactory<KeyType> keySerdeFactory,
ColumnSelectorFactory columnSelectorFactory,
AggregatorFactory[] aggregatorFactories,
int bufferGrouperMaxSize,
float bufferGrouperMaxLoadFactor,
int bufferGrouperInitialBuckets,
LimitedTemporaryStorage temporaryStorage,
com.fasterxml.jackson.databind.ObjectMapper spillMapper,
boolean spillingAllowed,
DefaultLimitSpec limitSpec,
boolean sortHasNonGroupingFields,
int mergeBufferSize) |
Modifier and Type | Method and Description |
---|---|
AggregateResult |
aggregate(KeyType key,
int keyHash)
Aggregate the current row with the provided key.
|
void |
close()
Close the grouper and release associated resources.
|
void |
init()
Initialize the grouper.
|
boolean |
isInitialized()
Check this grouper is initialized or not.
|
CloseableIterator<Grouper.Entry<KeyType>> |
iterator(boolean sorted)
Iterate through entries.
|
List<String> |
mergeAndGetDictionary()
Returns a dictionary of string keys added to this grouper.
|
void |
reset()
Reset the grouper to its initial state.
|
void |
setSpillingAllowed(boolean spillingAllowed) |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
aggregate, hashFunction
public SpillingGrouper(com.google.common.base.Supplier<ByteBuffer> bufferSupplier, Grouper.KeySerdeFactory<KeyType> keySerdeFactory, ColumnSelectorFactory columnSelectorFactory, AggregatorFactory[] aggregatorFactories, int bufferGrouperMaxSize, float bufferGrouperMaxLoadFactor, int bufferGrouperInitialBuckets, LimitedTemporaryStorage temporaryStorage, com.fasterxml.jackson.databind.ObjectMapper spillMapper, boolean spillingAllowed, DefaultLimitSpec limitSpec, boolean sortHasNonGroupingFields, int mergeBufferSize)
public void init()
Grouper
Grouper.aggregate(Object)
and Grouper.aggregate(Object, int)
.public boolean isInitialized()
Grouper
isInitialized
in interface Grouper<KeyType>
public AggregateResult aggregate(KeyType key, int keyHash)
Grouper
aggregate
in interface Grouper<KeyType>
key
- key objectkeyHash
- result of Grouper.hashFunction()
on the keypublic void reset()
Grouper
public void close()
Grouper
public List<String> mergeAndGetDictionary()
public void setSpillingAllowed(boolean spillingAllowed)
public CloseableIterator<Grouper.Entry<KeyType>> iterator(boolean sorted)
Grouper
Some implementations allow writes even after this method is called. After you are done with the iterator
returned by this method, you should either call Grouper.close()
(if you are done with the Grouper) or
Grouper.reset()
(if you want to reuse it). Some implementations allow calling Grouper.iterator(boolean)
again if
you want another iterator. But, this method must not be called by multiple threads concurrently.
If "sorted" is true then the iterator will return sorted results. It will use KeyType's natural ordering on
deserialized objects, and will use the KeySerde#comparator()
on serialized objects. Woe be unto you
if these comparators are not equivalent.
Callers must process and discard the returned Grouper.Entry
s immediately because some implementations can reuse the
key objects.
Copyright © 2011–2018. All rights reserved.