io.druid.indexing.common.task

Class HadoopIndexTask

java.lang.Object

io.druid.indexing.common.task.AbstractTask

io.druid.indexing.common.task.HadoopTask

io.druid.indexing.common.task.HadoopIndexTask

All Implemented Interfaces:

Task

public class HadoopIndexTask
extends HadoopTask

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HadoopIndexTask.HadoopDetermineConfigInnerProcessing Called indirectly in run(TaskToolbox).
static class	HadoopIndexTask.HadoopIndexGeneratorInnerProcessing Called indirectly in run(TaskToolbox).

Field Summary

Fields inherited from class io.druid.indexing.common.task.HadoopTask

IS_DRUID_URL

Constructor Summary

Constructors

Constructor and Description
HadoopIndexTask(String id, HadoopIngestionSpec spec, String hadoopCoordinates, List<String> hadoopDependencyCoordinates, String classpathPrefix, com.fasterxml.jackson.databind.ObjectMapper jsonMapper, Map<String,Object> context)

Method Summary

Modifier and Type	Method and Description
String	getClasspathPrefix() Returns an extra classpath that should be prepended to the default classpath when running this task.
List<String>	getHadoopDependencyCoordinates()
int	getPriority() Returns task priority.
HadoopIngestionSpec	getSpec()
String	getType() Returns a descriptive label for this task type.
boolean	isReady(TaskActionClient taskActionClient) Execute preflight actions for a task.
TaskStatus	run(TaskToolbox toolbox) Execute a task.

Methods inherited from class io.druid.indexing.common.task.HadoopTask

buildClassLoader, buildClassLoader, invokeForeignLoader

Methods inherited from class io.druid.indexing.common.task.AbstractTask

canRestore, equals, getContext, getDataSource, getGroupId, getId, getNodeType, getQueryRunner, getTaskResource, hashCode, stopGracefully, success, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.druid.indexing.common.task.Task

getContextValue, getContextValue

Constructor Detail

HadoopIndexTask

public HadoopIndexTask(String id,
                       HadoopIngestionSpec spec,
                       String hadoopCoordinates,
                       List<String> hadoopDependencyCoordinates,
                       String classpathPrefix,
                       com.fasterxml.jackson.databind.ObjectMapper jsonMapper,
                       Map<String,Object> context)

Parameters:

spec - is used by the HadoopDruidIndexerJob to set up the appropriate parameters for creating Druid index segments. It may be modified.

Here, we will ensure that the DbConnectorConfig field of the spec is set to null, such that the job does not push a list of published segments the database. Instead, we will use the method IndexGeneratorJob.getPublishedSegments() to simply return a list of the published segments, and let the indexing service report these segments to the database.

Method Detail

getPriority

public int getPriority()

Description copied from interface: Task

Returns task priority. The task priority is currently used only for prioritized locking, but, in the future, it can be used for task scheduling, cluster resource management, etc.

Returns:

task priority

See Also:

for default task priorities

getType

public String getType()

Description copied from interface: Task

Returns a descriptive label for this task type. Used for metrics emission and logging.

Returns:

task type label

isReady

public boolean isReady(TaskActionClient taskActionClient)
                throws Exception

Description copied from interface: Task

Execute preflight actions for a task. This can be used to acquire locks, check preconditions, and so on. The actions must be idempotent, since this method may be executed multiple times. This typically runs on the coordinator. If this method throws an exception, the task should be considered a failure.

This method must be idempotent, as it may be run multiple times per task.

Parameters:

taskActionClient - action client for this task (not the full toolbox)

Returns:

true if ready, false if not ready yet

Throws:

Exception - if the task should be considered a failure

getSpec

public HadoopIngestionSpec getSpec()

getHadoopDependencyCoordinates

public List<String> getHadoopDependencyCoordinates()

Overrides:

getHadoopDependencyCoordinates in class HadoopTask

getClasspathPrefix

public String getClasspathPrefix()

Description copied from interface: Task

Returns an extra classpath that should be prepended to the default classpath when running this task. If no extra classpath should be prepended, this should return null or the empty string.

Specified by:

getClasspathPrefix in interface Task

Overrides:

getClasspathPrefix in class AbstractTask

run

public TaskStatus run(TaskToolbox toolbox)
               throws Exception

Description copied from interface: Task

Execute a task. This typically runs on a worker as determined by a TaskRunner, and will be run while holding a lock on our dataSource and implicit lock interval (if any). If this method throws an exception, the task should be considered a failure.

Parameters:

toolbox - Toolbox for this task

Returns:

Some kind of finished status (isRunnable must be false).

Throws:

Exception - if this task failed