DirectLoadPipelineStep

@Singleton

class DirectLoadPipelineStep<S : DirectLoader>(val directLoaderFactory: DirectLoaderFactory<S>, val accumulator: DirectLoadRecordAccumulator<S, StreamKey>, val taskFactory: LoadPipelineStepTaskFactory, @Named(value = "numInputPartitions") numInputPartitions: Int) : LoadPipelineStep

Constructors

constructor(directLoaderFactory: DirectLoaderFactory<S>, accumulator: DirectLoadRecordAccumulator<S, StreamKey>, taskFactory: LoadPipelineStepTaskFactory, @Named(value = "numInputPartitions") numInputPartitions: Int)

Enforce that each worker can hold at least one open loader. (TODO: Maybe clamp the number of input partitions instead?)

open override val numWorkers: Int

open override fun taskForPartition(partition: Int): LoadPipelineStepTask<*, *, *, *, *>