io.druid.storage.s3

Class S3Utils

java.lang.Object

io.druid.storage.s3.S3Utils

public class S3Utils
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static com.google.common.base.Predicate<Throwable>	S3RETRY

Constructor Summary

Constructors

Constructor and Description
S3Utils()

Method Summary

Modifier and Type	Method and Description
static void	closeStreamsQuietly(org.jets3t.service.model.S3Object s3Obj)
static String	constructSegmentPath(String baseKey, String storageDir)
static String	descriptorPathForSegmentPath(String s3Path)
static String	indexZipForSegmentPath(String s3Path)
static boolean	isObjectInBucket(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client, String bucketName, String objectKey)
static boolean	isServiceExceptionRecoverable(org.jets3t.service.ServiceException ex)
static <T> T	retryS3Operation(Callable<T> f) Retries S3 operations that fail due to io-related exceptions.
static Iterator<org.jets3t.service.model.StorageObject>	storageObjectsIterator(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client, String bucket, String prefix, long maxListingLength)
static String	toFilename(String key)
static String	toFilename(String key, String suffix)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

S3RETRY

public static final com.google.common.base.Predicate<Throwable> S3RETRY

Constructor Detail

S3Utils

public S3Utils()

Method Detail

closeStreamsQuietly

public static void closeStreamsQuietly(org.jets3t.service.model.S3Object s3Obj)

isServiceExceptionRecoverable

public static boolean isServiceExceptionRecoverable(org.jets3t.service.ServiceException ex)

retryS3Operation

public static <T> T retryS3Operation(Callable<T> f)
                              throws Exception

Retries S3 operations that fail due to io-related exceptions. Service-level exceptions (access denied, file not found, etc) are not retried.

Throws:

Exception

isObjectInBucket

public static boolean isObjectInBucket(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client,
                                       String bucketName,
                                       String objectKey)
                                throws org.jets3t.service.ServiceException

Throws:

org.jets3t.service.ServiceException

storageObjectsIterator

public static Iterator<org.jets3t.service.model.StorageObject> storageObjectsIterator(org.jets3t.service.impl.rest.httpclient.RestS3Service s3Client,
                                                                                      String bucket,
                                                                                      String prefix,
                                                                                      long maxListingLength)

constructSegmentPath

public static String constructSegmentPath(String baseKey,
                                          String storageDir)

descriptorPathForSegmentPath

public static String descriptorPathForSegmentPath(String s3Path)

indexZipForSegmentPath

public static String indexZipForSegmentPath(String s3Path)

toFilename

public static String toFilename(String key)

toFilename

public static String toFilename(String key,
                                String suffix)