io.druid.segment.realtime.appenderator

Class BatchAppenderatorDriver

java.lang.Object

io.druid.segment.realtime.appenderator.BaseAppenderatorDriver

io.druid.segment.realtime.appenderator.BatchAppenderatorDriver

All Implemented Interfaces:

Closeable, AutoCloseable

public class BatchAppenderatorDriver
extends BaseAppenderatorDriver

This class is specifialized for batch ingestion. In batch ingestion, the segment lifecycle is like:

 APPENDING -> PUSHED_AND_DROPPED -> PUBLISHED
 

• APPENDING: Segment is available for appending.
• PUSHED_AND_DROPPED: Segment is pushed to deep storage and dropped from the local storage.
• PUBLISHED: Segment's metadata is published to metastore.

Field Summary

Fields inherited from class io.druid.segment.realtime.appenderator.BaseAppenderatorDriver

appenderator, executor, segments

Constructor Summary

Constructors

Constructor and Description
BatchAppenderatorDriver(Appenderator appenderator, SegmentAllocator segmentAllocator, UsedSegmentChecker usedSegmentChecker) Create a driver.

Method Summary

Modifier and Type	Method and Description
AppenderatorDriverAddResult	add(InputRow row, String sequenceName) Add a row.
com.google.common.util.concurrent.ListenableFuture<SegmentsAndMetadata>	publishAll(TransactionalSegmentPublisher publisher) Publish all segments.
SegmentsAndMetadata	pushAllAndClear(long pushAndClearTimeoutMs) Push and drop all segments in the SegmentWithState.SegmentState.APPENDING state.
Object	startJob() This method always returns null because batch ingestion doesn't support restoring tasks on failures.

Methods inherited from class io.druid.segment.realtime.appenderator.BaseAppenderatorDriver

append, clear, close

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchAppenderatorDriver

public BatchAppenderatorDriver(Appenderator appenderator,
                               SegmentAllocator segmentAllocator,
                               UsedSegmentChecker usedSegmentChecker)

Create a driver.

Parameters:

appenderator - appenderator

segmentAllocator - segment allocator

usedSegmentChecker - used segment checker

Method Detail

startJob

@Nullable
public Object startJob()

This method always returns null because batch ingestion doesn't support restoring tasks on failures.

Specified by:

startJob in class BaseAppenderatorDriver

Returns:

always null

add

public AppenderatorDriverAddResult add(InputRow row,
                                       String sequenceName)
                                throws IOException

Add a row. Must not be called concurrently from multiple threads.

Parameters:

row - the row to add

sequenceName - sequenceName for this row's segment

Returns:

AppenderatorDriverAddResult

Throws:

IOException - if there is an I/O error while allocating or writing to a segment

pushAllAndClear

public SegmentsAndMetadata pushAllAndClear(long pushAndClearTimeoutMs)
                                    throws InterruptedException,
                                           ExecutionException,
                                           TimeoutException

Push and drop all segments in the SegmentWithState.SegmentState.APPENDING state.

Parameters:

pushAndClearTimeoutMs - timeout for pushing and dropping segments

Returns:

SegmentsAndMetadata for pushed and dropped segments

Throws:

InterruptedException

ExecutionException

TimeoutException

publishAll

public com.google.common.util.concurrent.ListenableFuture<SegmentsAndMetadata> publishAll(TransactionalSegmentPublisher publisher)

Publish all segments.

Parameters:

publisher - segment publisher

Returns:

a ListenableFuture for the publish task