io.druid.storage.hdfs

Class HdfsDataSegmentFinder

java.lang.Object

io.druid.storage.hdfs.HdfsDataSegmentFinder

All Implemented Interfaces:

DataSegmentFinder

public class HdfsDataSegmentFinder
extends Object
implements DataSegmentFinder

Constructor Summary

Constructors

Constructor and Description
HdfsDataSegmentFinder(org.apache.hadoop.conf.Configuration config, com.fasterxml.jackson.databind.ObjectMapper mapper)

Method Summary

Modifier and Type	Method and Description
Set<DataSegment>	findSegments(String workingDirPathStr, boolean updateDescriptor) This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsDataSegmentFinder

@Inject
public HdfsDataSegmentFinder(org.apache.hadoop.conf.Configuration config,
                                     com.fasterxml.jackson.databind.ObjectMapper mapper)

Method Detail

findSegments

public Set<DataSegment> findSegments(String workingDirPathStr,
                                     boolean updateDescriptor)
                              throws SegmentLoadingException

Description copied from interface: DataSegmentFinder

This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder. If not, it should throw SegmentLoadingException to let the caller know that descriptor.json exists while index.zip doesn't. If a segment is found and updateDescriptor is set, then this method should update the loadSpec in descriptor.json to reflect the location from where it was found. After the search, this method should return the set of segments that were found.

Specified by:

findSegments in interface DataSegmentFinder

Parameters:

workingDirPathStr - the String representation of the working directory path

updateDescriptor - if true, update loadSpec in descriptor.json if loadSpec's location is different from where desciptor.json was found

Returns:

a set of segments that were found underneath workingDirPath

Throws:

SegmentLoadingException