Airbyte Kotlin Bulk CDK

0.1.92

bulk-cdk-core-extract/io.airbyte.cdk.read/FeedBootstrap

FeedBootstrap

sealed class FeedBootstrap<T : Feed>

FeedBootstrap is the input to a PartitionsCreatorFactory.

This object conveniently packages the StateManager singleton with the feed for which the PartitionsCreatorFactory is to operate on, eventually causing the emission of Airbyte RECORD messages for the Streams in the feed. For this purpose, FeedBootstrap provides StreamRecordConsumer instances which essentially provide a layer of caching over OutputConsumer, leveraging the fact that all records for a given stream share the same schema.

Inheritors

GlobalFeedBootstrap

StreamFeedBootstrap

Types

object Companion

EfficientStreamRecordConsumer

open inner class EfficientStreamRecordConsumer(val stream: Stream, val outputDataChannel: OutputConsumer = outputConsumer) : StreamRecordConsumer

Efficient implementation of StreamRecordConsumer.

JsonSocketEfficientStreamRecordConsumer

inner class JsonSocketEfficientStreamRecordConsumer(stream: Stream, outputer: OutputConsumer) : FeedBootstrap.EfficientStreamRecordConsumer<T>

ProtoEfficientStreamRecordConsumer

inner class ProtoEfficientStreamRecordConsumer(val stream: Stream, socketProtobufOutputConsumer: SocketProtobufOutputConsumer) : StreamRecordConsumer

Properties

bufferByteSizeThresholdForFlush

val bufferByteSizeThresholdForFlush: Int

val clock: Clock

val currentState: OpaqueStateValue?

Convenience getter for the current state value for this feed.

dataChannelFormat

val dataChannelFormat: DataChannelFormat

dataChannelMedium

val dataChannelMedium: DataChannelMedium

val feed: T

Feed to emit records for.

val feeds: List<Feed>

Delegates to StateManager.feeds.

metaFieldDecorator

val metaFieldDecorator: MetaFieldDecorator

The MetaFieldDecorator instance which StreamRecordConsumer will use to decorate records.

val outputConsumer: StandardOutputConsumer

The OutputConsumer instance to which StreamRecordConsumer will delegate to.

Functions

fun currentState(feed: Feed): OpaqueStateValue?

Deletages to StateManager to return the current state value for any Feed.

Resets the state value of this feed and the streams in it to zero.

streamJsonSocketRecordConsumers

fun streamJsonSocketRecordConsumers(socketJsonOutputConsumer: SocketJsonOutputConsumer): Map<StreamIdentifier, FeedBootstrap.JsonSocketEfficientStreamRecordConsumer<T>>

streamProtoRecordConsumers

fun streamProtoRecordConsumers(socketProtoOutputConsumer: SocketProtobufOutputConsumer): Map<StreamIdentifier, FeedBootstrap.ProtoEfficientStreamRecordConsumer<T>>

streamRecordConsumers

fun streamRecordConsumers(): Map<StreamIdentifier, StreamRecordConsumer>

A map of all StreamRecordConsumer for this feed.