io.druid.data.input.impl

Class FileIteratingFirehose

java.lang.Object

io.druid.data.input.impl.FileIteratingFirehose

All Implemented Interfaces:

Firehose, Closeable, AutoCloseable

public class FileIteratingFirehose
extends Object
implements Firehose

Constructor Summary

Constructors

Constructor and Description
FileIteratingFirehose(Iterator<org.apache.commons.io.LineIterator> lineIterators, StringInputRowParser parser)
FileIteratingFirehose(Iterator<org.apache.commons.io.LineIterator> lineIterators, StringInputRowParser parser, Closeable closer)

Method Summary

Modifier and Type	Method and Description
void	close()
Runnable	commit() Returns a runnable that will "commit" everything read up to the point at which commit() is called.
boolean	hasMore() Returns whether there are more rows to process.
InputRow	nextRow() The next row available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileIteratingFirehose

public FileIteratingFirehose(Iterator<org.apache.commons.io.LineIterator> lineIterators,
                             StringInputRowParser parser)

FileIteratingFirehose

public FileIteratingFirehose(Iterator<org.apache.commons.io.LineIterator> lineIterators,
                             StringInputRowParser parser,
                             Closeable closer)

Method Detail

hasMore

public boolean hasMore()

Description copied from interface: Firehose

Returns whether there are more rows to process. This is used to indicate that another item is immediately available via ##nextRow(). Thus, if the stream is still available but there are no new messages on it, this call should block until a new message is available. If something happens such that the stream is no longer available, this should return false.

Specified by:

hasMore in interface Firehose

Returns:

true if and when there is another row available, false if the stream has dried up

nextRow

@Nullable
public InputRow nextRow()

Description copied from interface: Firehose

The next row available. Should only be called if hasMore returns true. The return value can be null which means the caller must skip this row.

Specified by:

nextRow in interface Firehose

Returns:

The next row

commit

public Runnable commit()

Description copied from interface: Firehose

Returns a runnable that will "commit" everything read up to the point at which commit() is called. This is often equivalent to everything that has been read since the last commit() call (or instantiation of the object), but doesn't necessarily have to be. This method is called when the main processing loop starts to persist its current batch of things to process. The returned runnable will be run when the current batch has been successfully persisted, there is usually some time lag between when this method is called and when the runnable is run. The Runnable is also run on a separate thread so its operation should be thread-safe. The Runnable is essentially just a lambda/closure that is run() after data supplied by this instance has been committed on the writer side of this interface protocol.

A simple implementation of this interface might do nothing when run() is called (in which case the same do-nothing instance can be returned every time), or a more complex implementation might clean up temporary resources that are no longer needed because of InputRows delivered by prior calls to ##nextRow().

Specified by:

commit in interface Firehose

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException