bulk-cdk-toolkit-load-iceberg-parquet/io.airbyte.cdk.load.toolkits.iceberg.parquet.io/BaseDeltaTaskWriter

BaseDeltaTaskWriter

abstract class BaseDeltaTaskWriter : BaseTaskWriter<T>

Implementation of the Iceberg BaseTaskWriter that handles delta-based updates (insert, update, delete).

N.B.: This class is implemented in Java due to visibility of the BaseEqualityDeltaWriter. That class is marked as protected, which is less visible than the BaseTaskWriter class or any other subclasses. This is not allowed in Kotlin, so this class is implemented in Java where that it is allowed. This class should not be converted to Kotlin while the visibility is still lesser in the super class.

Inheritors

PartitionedDeltaWriter

UnpartitionedDeltaWriter

Constructors

BaseDeltaTaskWriter

constructor(table: Table, spec: PartitionSpec, format: FileFormat, appenderFactory: FileAppenderFactory<Record>, fileFactory: OutputFileFactory, io: FileIO, targetFileSize: Long, schema: Schema, identifierFieldIds: Set<Integer>)

Types

RowDataDeltaWriter

open inner class RowDataDeltaWriter : BaseTaskWriter.BaseEqualityDeltaWriter

Custom writer implementation that supports inserting and deleting records based on equality.

Properties

NULL_PK_ERROR_MESSAGE

val NULL_PK_ERROR_MESSAGE: String = "Detected null value in primary key. The Iceberg protocol disallows this. This is either a bug in the source, or you should use the append/overwrite sync mode."

Functions

abort

open fun abort()

abstract fun close()

complete

open fun complete(): WriteResult

dataFiles

open fun dataFiles(): Array<DataFile>

route

abstract fun route(row: Record): BaseDeltaTaskWriter.RowDataDeltaWriter

wrapper

open fun wrapper(): InternalRecordWrapper

write

open fun write(row: Record)