bulk-cdk-core-load/io.airbyte.cdk.load.pipeline/DirectLoadPipeline

DirectLoadPipeline

@Singleton

@Requires(bean = DirectLoaderFactory::class)

class DirectLoadPipeline(val pipelineStep: DirectLoadPipelineStep<*>) : LoadPipeline

Used internally by the CDK to implement the DirectLoader.

Creates a single pipeline step reading from a (possibly partitioned) record stream. Batch updates are written to the batchStateUpdateQueue whenever the loader returns

Constructors

DirectLoadPipeline

constructor(pipelineStep: DirectLoadPipelineStep<*>)

Properties

pipelineStep

val pipelineStep: DirectLoadPipelineStep<*>

Functions

start

suspend fun start(launcher: suspend (Task) -> Unit)

stop

open suspend fun stop()

For closing intermediate queues or other resources.