io.druid.indexing.kafka

Class KafkaIndexTask

java.lang.Object

io.druid.indexing.common.task.AbstractTask

io.druid.indexing.kafka.KafkaIndexTask

All Implemented Interfaces:

Task, ChatHandler

public class KafkaIndexTask
extends AbstractTask
implements ChatHandler

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	KafkaIndexTask.Status

Field Summary

Fields

Modifier and Type	Field and Description
static long	PAUSE_FOREVER

Constructor Summary

Constructors

Constructor and Description
KafkaIndexTask(String id, TaskResource taskResource, DataSchema dataSchema, KafkaTuningConfig tuningConfig, KafkaIOConfig ioConfig, Map<String,Object> context, ChatHandlerProvider chatHandlerProvider)

Method Summary

Modifier and Type	Method and Description
boolean	canRestore() Returns whether or not this task can restore its progress from its on-disk working directory.
Map<Integer,Long>	getCurrentOffsets()
DataSchema	getDataSchema()
Map<Integer,Long>	getEndOffsets()
KafkaIOConfig	getIOConfig()
<T> QueryRunner<T>	getQueryRunner(Query<T> query) Returns query runners for this task.
org.joda.time.DateTime	getStartTime()
KafkaIndexTask.Status	getStatus()
KafkaTuningConfig	getTuningConfig()
String	getType() Returns a descriptive label for this task type.
boolean	isReady(TaskActionClient taskActionClient) Execute preflight actions for a task.
javax.ws.rs.core.Response	pause(long timeout) Signals the ingestion loop to pause.
void	resume()
TaskStatus	run(TaskToolbox toolbox) Execute a task.
javax.ws.rs.core.Response	setEndOffsets(Map<Integer,Long> offsets, boolean resume)
void	stopGracefully() Asks a task to arrange for its "run" method to exit promptly.

Methods inherited from class io.druid.indexing.common.task.AbstractTask

equals, getClasspathPrefix, getContext, getContextValue, getDataSource, getGroupId, getId, getNodeType, getTaskLocks, getTaskResource, hashCode, joinId, makeId, success, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

PAUSE_FOREVER

public static final long PAUSE_FOREVER

See Also:

Constant Field Values

Constructor Detail

KafkaIndexTask

public KafkaIndexTask(String id,
                      TaskResource taskResource,
                      DataSchema dataSchema,
                      KafkaTuningConfig tuningConfig,
                      KafkaIOConfig ioConfig,
                      Map<String,Object> context,
                      ChatHandlerProvider chatHandlerProvider)

Method Detail

getType

public String getType()

Description copied from interface: Task

Returns a descriptive label for this task type. Used for metrics emission and logging.

Specified by:

getType in interface Task

Returns:

task type label

isReady

public boolean isReady(TaskActionClient taskActionClient)
                throws Exception

Description copied from interface: Task

Execute preflight actions for a task. This can be used to acquire locks, check preconditions, and so on. The actions must be idempotent, since this method may be executed multiple times. This typically runs on the coordinator. If this method throws an exception, the task should be considered a failure.

This method must be idempotent, as it may be run multiple times per task.

Specified by:

isReady in interface Task

Parameters:

taskActionClient - action client for this task (not the full toolbox)

Returns:

true if ready, false if not ready yet

Throws:

Exception - if the task should be considered a failure

getDataSchema

public DataSchema getDataSchema()

getTuningConfig

public KafkaTuningConfig getTuningConfig()

getIOConfig

public KafkaIOConfig getIOConfig()

run

public TaskStatus run(TaskToolbox toolbox)
               throws Exception

Description copied from interface: Task

Execute a task. This typically runs on a worker as determined by a TaskRunner, and will be run while holding a lock on our dataSource and implicit lock interval (if any). If this method throws an exception, the task should be considered a failure.

Specified by:

run in interface Task

Parameters:

toolbox - Toolbox for this task

Returns:

Some kind of finished status (isRunnable must be false).

Throws:

Exception - if this task failed

canRestore

public boolean canRestore()

Description copied from interface: Task

Returns whether or not this task can restore its progress from its on-disk working directory. Restorable tasks may be started with a non-empty working directory. Tasks that exit uncleanly may still have a chance to attempt restores, meaning that restorable tasks should be able to deal with potentially partially written on-disk state.

Specified by:

canRestore in interface Task

Overrides:

canRestore in class AbstractTask

stopGracefully

public void stopGracefully()

Description copied from interface: Task

Asks a task to arrange for its "run" method to exit promptly. This method will only be called if Task.canRestore() returns true. Tasks that take too long to stop gracefully will be terminated with extreme prejudice.

Specified by:

stopGracefully in interface Task

Overrides:

stopGracefully in class AbstractTask

getQueryRunner

public <T> QueryRunner<T> getQueryRunner(Query<T> query)

Description copied from interface: Task

Returns query runners for this task. If this task is not meant to answer queries over its datasource, this method should return null.

Specified by:

getQueryRunner in interface Task

Overrides:

getQueryRunner in class AbstractTask

Type Parameters:

T - query result type

Returns:

query runners for this task

getStatus

public KafkaIndexTask.Status getStatus()

getCurrentOffsets

public Map<Integer,Long> getCurrentOffsets()

getEndOffsets

public Map<Integer,Long> getEndOffsets()

setEndOffsets

public javax.ws.rs.core.Response setEndOffsets(Map<Integer,Long> offsets,
                                               boolean resume)
                                        throws InterruptedException

Throws:

InterruptedException

pause

public javax.ws.rs.core.Response pause(long timeout)
                                throws InterruptedException

Signals the ingestion loop to pause.

Parameters:

timeout - how long to pause for before resuming in milliseconds, <= 0 means indefinitely

Returns:

one of the following Responses: 400 Bad Request if the task has started publishing; 202 Accepted if the method has timed out and returned before the task has paused; 200 OK with a map of the current partition offsets in the response body if the task successfully paused

Throws:

InterruptedException

resume

public void resume()
            throws InterruptedException

Throws:

InterruptedException

getStartTime

public org.joda.time.DateTime getStartTime()