StreamManager

class StreamManager(val stream: DestinationStream)

Manages the state of a single stream.

Constructors

Link copied to clipboard
constructor(stream: DestinationStream)

Properties

Link copied to clipboard

Functions

Link copied to clipboard

True if persisted counts associated with the index checkpointId are equal to the number of records read.

Link copied to clipboard

Suspend until the stream completes, returning the result.

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard

Some destinations need to perform expensive post-processing at the end of a sync, but can skip that post-processing if the sync had 0 records. So we should tell the destination whether any records were processed.

Link copied to clipboard
fun incrementByteCount(bytes: Long, checkpointId: CheckpointId): Long
Link copied to clipboard
Link copied to clipboard

Count incoming record and return the record's index. If markEndOfStream has been called, this should throw an exception.

Link copied to clipboard

True if the stream processing has not yet been marked as successful or failed.

Link copied to clipboard

True if all records in the stream have been marked as completed AND the stream has been marked as complete.

Link copied to clipboard
fun markEndOfStream(receivedStreamCompleteMessage: Boolean): Long

Mark the end-of-stream, set the end of stream variant (complete or incomplete) and return the record count. Expect this exactly once. Expect no further countRecordIn, and expect that markProcessingSucceeded will always occur after this, while markProcessingFailed can occur before or after.

Link copied to clipboard

Indicates destination processing of the stream failed. Returns false if task was already complete

Link copied to clipboard

Indicates destination processing of the stream succeeded, regardless of complete/incomplete status. This should only be called after all records and end of stream messages have been read.

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard

Whether we received a stream complete message for the managed stream.

Link copied to clipboard

Mark this stream manager as having initiated a terminal task (i.e. CloseStreamTask or FailStreamTask).