bulk-cdk-toolkit-load-object-storage/io.airbyte.cdk.load.pipline.object_storage/ObjectLoaderFormattedPartPartitioner

ObjectLoaderFormattedPartPartitioner

class ObjectLoaderFormattedPartPartitioner<K : WithStream, T> : OutputPartitioner<K, T, ObjectKey, ObjectLoaderPartFormatter.FormattedPart>

Distribute the parts randomly across loaders. (Testing shows this is the most efficient pattern.)

Constructors

Link copied to clipboard
constructor()

Functions

Link copied to clipboard
open override fun getOutputKey(inputKey: K, output: ObjectLoaderPartFormatter.FormattedPart): ObjectKey
Link copied to clipboard
open override fun getPart(outputKey: ObjectKey, inputPart: Int, numParts: Int): Int