io.druid.storage.hdfs

Class HdfsFileTimestampVersionFinder

java.lang.Object

io.druid.storage.hdfs.HdfsDataSegmentPuller

io.druid.storage.hdfs.HdfsFileTimestampVersionFinder

All Implemented Interfaces:

SearchableVersionedDataFinder<URI>, io.druid.segment.loading.DataSegmentPuller, io.druid.segment.loading.URIDataPuller

public class HdfsFileTimestampVersionFinder
extends HdfsDataSegmentPuller
implements SearchableVersionedDataFinder<URI>

This is implemented explicitly for URIExtractionNamespaceFunctionFactory If you have a use case for this interface beyond URIExtractionNamespaceFunctionFactory please bring it up in the dev list.

Nested Class Summary

Nested classes/interfaces inherited from class io.druid.storage.hdfs.HdfsDataSegmentPuller

HdfsDataSegmentPuller.HdfsIOException

Field Summary

Fields inherited from class io.druid.storage.hdfs.HdfsDataSegmentPuller

config, DEFAULT_RETRY_COUNT

Constructor Summary

Constructors

Constructor and Description
HdfsFileTimestampVersionFinder(org.apache.hadoop.conf.Configuration config)

Method Summary

Methods

Modifier and Type	Method and Description
Class<URI>	getDataDescriptorClass()
URI	getLatestVersion(URI uri, Pattern pattern) Returns the latest modified file at the uri of interest.

Methods inherited from class io.druid.storage.hdfs.HdfsDataSegmentPuller

buildFileObject, buildFileObject, getInputStream, getInputStream, getSegmentFiles, getSegmentFiles, getSegmentFiles, getVersion, shouldRetryPredicate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsFileTimestampVersionFinder

@Inject
public HdfsFileTimestampVersionFinder(org.apache.hadoop.conf.Configuration config)

Method Detail

getLatestVersion

public URI getLatestVersion(URI uri,
                   Pattern pattern)

Returns the latest modified file at the uri of interest.

Specified by:

getLatestVersion in interface SearchableVersionedDataFinder<URI>

Parameters:

uri - Either a directory or a file on HDFS. If it is a file, the parent directory will be searched.

pattern - A pattern matcher for file names in the directory of interest. Passing `null` results in matching any file in the directory.

Returns:

The URI of the file with the most recent modified timestamp.

getDataDescriptorClass

public Class<URI> getDataDescriptorClass()

Specified by:

getDataDescriptorClass in interface SearchableVersionedDataFinder<URI>

Returns:

The class of the descriptor for the data