io.druid.segment.realtime.plumber

Class RealtimePlumber

java.lang.Object

io.druid.segment.realtime.plumber.RealtimePlumber

All Implemented Interfaces:

Plumber

Direct Known Subclasses:

FlushingPlumber

public class RealtimePlumber
extends Object
implements Plumber

Constructor Summary

Constructors

Constructor and Description
RealtimePlumber(DataSchema schema, RealtimeTuningConfig config, FireDepartmentMetrics metrics, com.metamx.emitter.service.ServiceEmitter emitter, QueryRunnerFactoryConglomerate conglomerate, DataSegmentAnnouncer segmentAnnouncer, ExecutorService queryExecutorService, io.druid.segment.loading.DataSegmentPusher dataSegmentPusher, SegmentPublisher segmentPublisher, FilteredServerView serverView)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	abandonSegment(long truncatedTime, Sink sink) Unannounces a given sink and removes all local references to it.
int	add(io.druid.data.input.InputRow row)
protected void	bootstrapSinksFromDisk()
protected File	computeBaseDir(DataSchema schema)
protected File	computePersistDir(DataSchema schema, org.joda.time.Interval interval)
void	finishJob() Perform any final processing and clean up after ourselves.
RealtimeTuningConfig	getConfig()
<T> QueryRunner<T>	getQueryRunner(Query<T> query)
RejectionPolicy	getRejectionPolicy()
DataSchema	getSchema()
Sink	getSink(long timestamp)
Map<Long,Sink>	getSinks()
protected void	initializeExecutors()
void	persist(Runnable commitRunnable) Persist any in-memory indexed data to durable storage.
protected int	persistHydrant(FireHydrant indexToPersist, DataSchema schema, org.joda.time.Interval interval) Persists the given hydrant and returns the number of rows persisted
protected void	shutdownExecutors()
void	startJob() Perform any initial setup.
protected void	startPersistThread()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RealtimePlumber

public RealtimePlumber(DataSchema schema,
               RealtimeTuningConfig config,
               FireDepartmentMetrics metrics,
               com.metamx.emitter.service.ServiceEmitter emitter,
               QueryRunnerFactoryConglomerate conglomerate,
               DataSegmentAnnouncer segmentAnnouncer,
               ExecutorService queryExecutorService,
               io.druid.segment.loading.DataSegmentPusher dataSegmentPusher,
               SegmentPublisher segmentPublisher,
               FilteredServerView serverView)

Method Detail

getSchema

public DataSchema getSchema()

getConfig

public RealtimeTuningConfig getConfig()

getRejectionPolicy

public RejectionPolicy getRejectionPolicy()

getSinks

public Map<Long,Sink> getSinks()

startJob

public void startJob()

Description copied from interface: Plumber

Perform any initial setup. Should be called before using any other methods, and should be paired with a corresponding call to Plumber.finishJob().

Specified by:

startJob in interface Plumber

add

public int add(io.druid.data.input.InputRow row)
        throws IndexSizeExceededException

Specified by:

add in interface Plumber

Parameters:

row - - the row to insert

Returns:

- positive numbers indicate how many summarized rows exist in the index for that timestamp, -1 means a row was thrown away because it was too late

Throws:

IndexSizeExceededException

getSink

public Sink getSink(long timestamp)

Specified by:

getSink in interface Plumber

getQueryRunner

public <T> QueryRunner<T> getQueryRunner(Query<T> query)

Specified by:

getQueryRunner in interface Plumber

persist

public void persist(Runnable commitRunnable)

Description copied from interface: Plumber

Persist any in-memory indexed data to durable storage. This may be only somewhat durable, e.g. the machine's local disk.

Specified by:

persist in interface Plumber

Parameters:

commitRunnable - code to run after persisting data

finishJob

public void finishJob()

Description copied from interface: Plumber

Perform any final processing and clean up after ourselves. Should be called after all data has been fed into sinks and persisted.

Specified by:

finishJob in interface Plumber

initializeExecutors

protected void initializeExecutors()

shutdownExecutors

protected void shutdownExecutors()

bootstrapSinksFromDisk

protected void bootstrapSinksFromDisk()

startPersistThread

protected void startPersistThread()

abandonSegment

protected void abandonSegment(long truncatedTime,
                  Sink sink)

Unannounces a given sink and removes all local references to it. It is important that this is only called from the single-threaded mergeExecutor, since otherwise chaos may ensue if merged segments are deleted while being created.

Parameters:

truncatedTime - sink key

sink - sink to unannounce

computeBaseDir

protected File computeBaseDir(DataSchema schema)

computePersistDir

protected File computePersistDir(DataSchema schema,
                     org.joda.time.Interval interval)

persistHydrant

protected int persistHydrant(FireHydrant indexToPersist,
                 DataSchema schema,
                 org.joda.time.Interval interval)

Persists the given hydrant and returns the number of rows persisted

Parameters:

indexToPersist - hydrant to persist

schema - datasource schema

interval - interval to persist

Returns:

the number of rows persisted