DirectLoadPipelineStep

@Singleton
@Requires(bean = DirectLoaderFactory::class)
class DirectLoadPipelineStep<S : DirectLoader>(val directLoaderFactory: DirectLoaderFactory<S>, val accumulator: DirectLoadRecordAccumulator<S, StreamKey>, val taskFactory: LoadPipelineStepTaskFactory, @Named(value = "numInputPartitions") numInputPartitions: Int) : LoadPipelineStep

Constructors

Link copied to clipboard
constructor(directLoaderFactory: DirectLoaderFactory<S>, accumulator: DirectLoadRecordAccumulator<S, StreamKey>, taskFactory: LoadPipelineStepTaskFactory, @Named(value = "numInputPartitions") numInputPartitions: Int)

Properties

Link copied to clipboard

Enforce that each worker can hold at least one open loader. (TODO: Maybe clamp the number of input partitions instead?)

Link copied to clipboard
open override val numWorkers: Int
Link copied to clipboard

Functions

Link copied to clipboard
open override fun taskForPartition(partition: Int): LoadPipelineStepTask<*, *, *, *, *>