io.druid.storage.s3

Class S3DataSegmentFinder

java.lang.Object

io.druid.storage.s3.S3DataSegmentFinder

All Implemented Interfaces:

DataSegmentFinder

public class S3DataSegmentFinder
extends Object
implements DataSegmentFinder

Constructor Summary

Constructors

Constructor and Description
S3DataSegmentFinder(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client, S3DataSegmentPusherConfig config, com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Summary

Modifier and Type	Method and Description
Set<DataSegment>	findSegments(String workingDirPath, boolean updateDescriptor) This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3DataSegmentFinder

@Inject
public S3DataSegmentFinder(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client,
                                   S3DataSegmentPusherConfig config,
                                   com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Detail

findSegments

public Set<DataSegment> findSegments(String workingDirPath,
                                     boolean updateDescriptor)
                              throws SegmentLoadingException

Description copied from interface: DataSegmentFinder

This method should first recursively look for descriptor.json (partitionNum_descriptor.json for HDFS data storage) underneath workingDirPath and then verify that index.zip (partitionNum_index.zip for HDFS data storage) exists in the same folder. If not, it should throw SegmentLoadingException to let the caller know that descriptor.json exists while index.zip doesn't. If a segment is found and updateDescriptor is set, then this method should update the loadSpec in descriptor.json to reflect the location from where it was found. After the search, this method should return the set of segments that were found.

Specified by:

findSegments in interface DataSegmentFinder

Parameters:

workingDirPath - the String representation of the working directory path

updateDescriptor - if true, update loadSpec in descriptor.json if loadSpec's location is different from where desciptor.json was found

Returns:

a set of segments that were found underneath workingDirPath

Throws:

SegmentLoadingException