io.druid.indexing.overlord

Class ThreadPoolTaskRunner

java.lang.Object

io.druid.indexing.overlord.ThreadPoolTaskRunner

All Implemented Interfaces:

TaskRunner, QuerySegmentWalker

public class ThreadPoolTaskRunner
extends Object
implements TaskRunner, QuerySegmentWalker

Runs tasks in a JVM thread using an ExecutorService.

Constructor Summary

Constructors

Constructor and Description
ThreadPoolTaskRunner(TaskToolboxFactory toolboxFactory, TaskConfig taskConfig, com.metamx.emitter.service.ServiceEmitter emitter, DruidNode node)

Method Summary

Modifier and Type	Method and Description
Collection<TaskRunnerWorkItem>	getKnownTasks()
Collection<TaskRunnerWorkItem>	getPendingTasks()
<T> QueryRunner<T>	getQueryRunnerForIntervals(Query<T> query, Iterable<org.joda.time.Interval> intervals) Gets the Queryable for a given interval, the Queryable returned can be any version(s) or partitionNumber(s) such that it represents the interval.
<T> QueryRunner<T>	getQueryRunnerForSegments(Query<T> query, Iterable<SegmentDescriptor> specs) Gets the Queryable for a given list of SegmentSpecs.
Collection<TaskRunnerWorkItem>	getRunningTasks()
com.google.common.base.Optional<ScalingStats>	getScalingStats() Some runners are able to scale up and down their capacity in a dynamic manner.
void	registerListener(TaskRunnerListener listener, Executor executor) Register a listener with this task runner.
List<Pair<Task,com.google.common.util.concurrent.ListenableFuture<TaskStatus>>>	restore() Some task runners can restart previously-running tasks after being bounced.
com.google.common.util.concurrent.ListenableFuture<TaskStatus>	run(Task task) Run a task.
void	shutdown(String taskid) Inform the task runner it can clean up any resources associated with a task.
void	start() Start the state of the runner
void	stop() Stop this task runner.
void	unregisterListener(String listenerId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ThreadPoolTaskRunner

@Inject
public ThreadPoolTaskRunner(TaskToolboxFactory toolboxFactory,
                                    TaskConfig taskConfig,
                                    com.metamx.emitter.service.ServiceEmitter emitter,
                                    DruidNode node)

Method Detail

restore

public List<Pair<Task,com.google.common.util.concurrent.ListenableFuture<TaskStatus>>> restore()

Description copied from interface: TaskRunner

Some task runners can restart previously-running tasks after being bounced. This method does that, and returns the list of tasks (and status futures).

Specified by:

restore in interface TaskRunner

registerListener

public void registerListener(TaskRunnerListener listener,
                             Executor executor)

Description copied from interface: TaskRunner

Register a listener with this task runner. On registration, the listener will get events corresponding to the current state of known tasks. Listener notifications are submitted to the executor in the order they occur, but it is up to the executor to decide when to actually run the notifications. If your listeners will not block, feel free to use a same-thread executor. Listeners that may block should use a separate executor, generally a single-threaded one with a fifo queue so the order of notifications is retained.

Specified by:

registerListener in interface TaskRunner

Parameters:

listener - the listener

executor - executor to run callbacks in

unregisterListener

public void unregisterListener(String listenerId)

Specified by:

unregisterListener in interface TaskRunner

stop

public void stop()

Description copied from interface: TaskRunner

Stop this task runner. This may block until currently-running tasks can be gracefully stopped. After calling stopping, "run" will not accept further tasks.

Specified by:

stop in interface TaskRunner

run

public com.google.common.util.concurrent.ListenableFuture<TaskStatus> run(Task task)

Description copied from interface: TaskRunner

Run a task. The returned status should be some kind of completed status.

Specified by:

run in interface TaskRunner

Parameters:

task - task to run

Returns:

task status, eventually

shutdown

public void shutdown(String taskid)

Description copied from interface: TaskRunner

Inform the task runner it can clean up any resources associated with a task. This implies shutdown of any currently-running tasks.

Specified by:

shutdown in interface TaskRunner

Parameters:

taskid - task ID to clean up resources for

getRunningTasks

public Collection<TaskRunnerWorkItem> getRunningTasks()

Specified by:

getRunningTasks in interface TaskRunner

getPendingTasks

public Collection<TaskRunnerWorkItem> getPendingTasks()

Specified by:

getPendingTasks in interface TaskRunner

getKnownTasks

public Collection<TaskRunnerWorkItem> getKnownTasks()

Specified by:

getKnownTasks in interface TaskRunner

getScalingStats

public com.google.common.base.Optional<ScalingStats> getScalingStats()

Description copied from interface: TaskRunner

Some runners are able to scale up and down their capacity in a dynamic manner. This returns stats on those activities

Specified by:

getScalingStats in interface TaskRunner

Returns:

ScalingStats if the runner has an underlying resource which can scale, Optional.absent() otherwise

start

public void start()

Description copied from interface: TaskRunner

Start the state of the runner

Specified by:

start in interface TaskRunner

getQueryRunnerForIntervals

public <T> QueryRunner<T> getQueryRunnerForIntervals(Query<T> query,
                                                     Iterable<org.joda.time.Interval> intervals)

Description copied from interface: QuerySegmentWalker

Gets the Queryable for a given interval, the Queryable returned can be any version(s) or partitionNumber(s) such that it represents the interval.

Specified by:

getQueryRunnerForIntervals in interface QuerySegmentWalker

Type Parameters:

T - query result type

Parameters:

query - the query to find a Queryable for

intervals - the intervals to find a Queryable for

Returns:

a Queryable object that represents the interval

getQueryRunnerForSegments

public <T> QueryRunner<T> getQueryRunnerForSegments(Query<T> query,
                                                    Iterable<SegmentDescriptor> specs)

Description copied from interface: QuerySegmentWalker

Gets the Queryable for a given list of SegmentSpecs.

Specified by:

getQueryRunnerForSegments in interface QuerySegmentWalker

Type Parameters:

T - the query result type

Parameters:

query - the query to return a Queryable for

specs - the list of SegmentSpecs to find a Queryable for

Returns:

the Queryable object with the given SegmentSpecs