io.druid.segment.loading

Interface DataSegmentFinder

All Known Implementing Classes:

HdfsDataSegmentFinder, LocalDataSegmentFinder, S3DataSegmentFinder

public interface DataSegmentFinder

A DataSegmentFinder is responsible for finding Druid segments underneath a specified directory and optionally updates all descriptor.json files on deep storage with correct loadSpec.

Method Summary

Modifier and Type	Method and Description
Set<DataSegment>	findSegments(String workingDirPath, boolean updateDescriptor) This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder.

Method Detail

findSegments

Set<DataSegment> findSegments(String workingDirPath,
                              boolean updateDescriptor)
                       throws SegmentLoadingException

This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder. If not, it should throw SegmentLoadingException to let the caller know that descriptor.json exists while index.zip doesn't. If a segment is found and updateDescriptor is set, then this method should update the loadSpec in descriptor.json to reflect the location from where it was found. After the search, this method should return the set of segments that were found.

Parameters:

workingDirPath - the String representation of the working directory path

updateDescriptor - if true, update loadSpec in descriptor.json if loadSpec's location is different from where desciptor.json was found

Returns:

a set of segments that were found underneath workingDirPath

Throws:

SegmentLoadingException