PartitionsCreator

A PartitionsCreator breaks down a Feed (a stream, or some global data feed) into zero, one or more partitions. Each partition is defined and read by a PartitionReader instance. These execute concurrently, but are joined serially because the state checkpoints need to appear in sequence.

  • zero partitions means that there is no more records to be read for this Feed;

  • one partition effectively means that the records will be read serially;

  • many partitions therefore involves some concurrency within the Feed.

Inheritors

Functions

Link copied to clipboard
abstract fun releaseResources()

Called after run to release any resources acquired by tryAcquireResources.

Link copied to clipboard
abstract suspend fun run(): List<PartitionReader>

Creates PartitionReader instances.

Link copied to clipboard

Called before run to try to acquire all the resources required for its execution. These may be, but are not limited to: