io.druid.indexing.overlord

Class ForkingTaskRunner

java.lang.Object

io.druid.indexing.overlord.ForkingTaskRunner

All Implemented Interfaces:

TaskRunner, TaskLogStreamer

public class ForkingTaskRunner
extends Object
implements TaskRunner, TaskLogStreamer

Runs tasks in separate processes using the "internal peon" verb.

Constructor Summary

Constructors

Constructor and Description
ForkingTaskRunner(ForkingTaskRunnerConfig config, TaskConfig taskConfig, WorkerConfig workerConfig, Properties props, TaskLogPusher taskLogPusher, com.fasterxml.jackson.databind.ObjectMapper jsonMapper, DruidNode node)

Method Summary

Modifier and Type	Method and Description
Collection<TaskRunnerWorkItem>	getKnownTasks()
Collection<TaskRunnerWorkItem>	getPendingTasks()
Collection<TaskRunnerWorkItem>	getRunningTasks()
com.google.common.base.Optional<ScalingStats>	getScalingStats() Some runners are able to scale up and down their capacity in a dynamic manner.
void	registerListener(TaskRunnerListener listener, Executor executor) Register a listener with this task runner.
List<Pair<Task,com.google.common.util.concurrent.ListenableFuture<TaskStatus>>>	restore() Some task runners can restart previously-running tasks after being bounced.
com.google.common.util.concurrent.ListenableFuture<TaskStatus>	run(Task task) Run a task.
void	shutdown(String taskid) Inform the task runner it can clean up any resources associated with a task.
void	start() Start the state of the runner
void	stop() Stop this task runner.
com.google.common.base.Optional<com.google.common.io.ByteSource>	streamTaskLog(String taskid, long offset) Stream log for a task.
void	unregisterListener(String listenerId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ForkingTaskRunner

@Inject
public ForkingTaskRunner(ForkingTaskRunnerConfig config,
                                 TaskConfig taskConfig,
                                 WorkerConfig workerConfig,
                                 Properties props,
                                 TaskLogPusher taskLogPusher,
                                 com.fasterxml.jackson.databind.ObjectMapper jsonMapper,
                                 DruidNode node)

Method Detail

restore

public List<Pair<Task,com.google.common.util.concurrent.ListenableFuture<TaskStatus>>> restore()

Description copied from interface: TaskRunner

Some task runners can restart previously-running tasks after being bounced. This method does that, and returns the list of tasks (and status futures).

Specified by:

restore in interface TaskRunner

registerListener

public void registerListener(TaskRunnerListener listener,
                             Executor executor)

Description copied from interface: TaskRunner

Register a listener with this task runner. On registration, the listener will get events corresponding to the current state of known tasks. Listener notifications are submitted to the executor in the order they occur, but it is up to the executor to decide when to actually run the notifications. If your listeners will not block, feel free to use a same-thread executor. Listeners that may block should use a separate executor, generally a single-threaded one with a fifo queue so the order of notifications is retained.

Specified by:

registerListener in interface TaskRunner

Parameters:

listener - the listener

executor - executor to run callbacks in

unregisterListener

public void unregisterListener(String listenerId)

Specified by:

unregisterListener in interface TaskRunner

run

public com.google.common.util.concurrent.ListenableFuture<TaskStatus> run(Task task)

Description copied from interface: TaskRunner

Run a task. The returned status should be some kind of completed status.

Specified by:

run in interface TaskRunner

Parameters:

task - task to run

Returns:

task status, eventually

stop

public void stop()

Description copied from interface: TaskRunner

Stop this task runner. This may block until currently-running tasks can be gracefully stopped. After calling stopping, "run" will not accept further tasks.

Specified by:

stop in interface TaskRunner

shutdown

public void shutdown(String taskid)

Description copied from interface: TaskRunner

Inform the task runner it can clean up any resources associated with a task. This implies shutdown of any currently-running tasks.

Specified by:

shutdown in interface TaskRunner

Parameters:

taskid - task ID to clean up resources for

getRunningTasks

public Collection<TaskRunnerWorkItem> getRunningTasks()

Specified by:

getRunningTasks in interface TaskRunner

getPendingTasks

public Collection<TaskRunnerWorkItem> getPendingTasks()

Specified by:

getPendingTasks in interface TaskRunner

getKnownTasks

public Collection<TaskRunnerWorkItem> getKnownTasks()

Specified by:

getKnownTasks in interface TaskRunner

getScalingStats

public com.google.common.base.Optional<ScalingStats> getScalingStats()

Description copied from interface: TaskRunner

Some runners are able to scale up and down their capacity in a dynamic manner. This returns stats on those activities

Specified by:

getScalingStats in interface TaskRunner

Returns:

ScalingStats if the runner has an underlying resource which can scale, Optional.absent() otherwise

start

public void start()

Description copied from interface: TaskRunner

Start the state of the runner

Specified by:

start in interface TaskRunner

streamTaskLog

public com.google.common.base.Optional<com.google.common.io.ByteSource> streamTaskLog(String taskid,
                                                                                      long offset)

Description copied from interface: TaskLogStreamer

Stream log for a task.

Specified by:

streamTaskLog in interface TaskLogStreamer

offset - If zero, stream the entire log. If positive, attempt to read from this position onwards. If negative, attempt to read this many bytes from the end of the file (like tail -n).

Returns:

input supplier for this log, if available from this provider