io.druid.storage.google

Class GoogleDataSegmentPuller

java.lang.Object

io.druid.storage.google.GoogleDataSegmentPuller

All Implemented Interfaces:

DataSegmentPuller, URIDataPuller

public class GoogleDataSegmentPuller
extends Object
implements DataSegmentPuller, URIDataPuller

Constructor Summary

Constructors

Constructor and Description
GoogleDataSegmentPuller(GoogleStorage storage)

Method Summary

Modifier and Type	Method and Description
InputStream	getInputStream(URI uri) Create a new InputStream based on the URI
void	getSegmentFiles(DataSegment segment, File outDir) Pull down segment files for the given DataSegment and put them in the given directory.
FileUtils.FileCopyResult	getSegmentFiles(String bucket, String path, File outDir)
String	getVersion(URI uri) Returns an abstract "version" for the URI.
void	prepareOutDir(File outDir)
com.google.common.base.Predicate<Throwable>	shouldRetryPredicate() Evaluates a Throwable to see if it is recoverable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GoogleDataSegmentPuller

@Inject
public GoogleDataSegmentPuller(GoogleStorage storage)

Method Detail

getSegmentFiles

public void getSegmentFiles(DataSegment segment,
                            File outDir)
                     throws SegmentLoadingException

Description copied from interface: DataSegmentPuller

Pull down segment files for the given DataSegment and put them in the given directory.

Specified by:

getSegmentFiles in interface DataSegmentPuller

Parameters:

segment - The segment to pull down files for

outDir - The directory to store the files in

Throws:

SegmentLoadingException - if there are any errors

getSegmentFiles

public FileUtils.FileCopyResult getSegmentFiles(String bucket,
                                                String path,
                                                File outDir)
                                         throws SegmentLoadingException

Throws:

SegmentLoadingException

prepareOutDir

public void prepareOutDir(File outDir)
                   throws ISE

Throws:

ISE

getInputStream

public InputStream getInputStream(URI uri)
                           throws IOException

Description copied from interface: URIDataPuller

Create a new InputStream based on the URI

Specified by:

getInputStream in interface URIDataPuller

Parameters:

uri - The URI to open an Input Stream to

Returns:

A new InputStream which streams the URI in question

Throws:

IOException

getVersion

public String getVersion(URI uri)
                  throws IOException

Description copied from interface: URIDataPuller

Returns an abstract "version" for the URI. The exact meaning of the version is left up to the implementation.

Specified by:

getVersion in interface URIDataPuller

Parameters:

uri - The URI to check

Returns:

A "version" as interpreted by the URIDataPuller implementation

Throws:

IOException - on error

shouldRetryPredicate

public com.google.common.base.Predicate<Throwable> shouldRetryPredicate()

Description copied from interface: URIDataPuller

Evaluates a Throwable to see if it is recoverable. This is expected to be used in conjunction with the other methods to determine if anything thrown from the method should be retried.

Specified by:

shouldRetryPredicate in interface URIDataPuller

Returns:

Predicate function indicating if the Throwable is recoverable