bulk-cdk-core-base/io.airbyte.cdk.output

Package-level declarations

Types

abstract class BaseStdoutOutputConsumer(clock: Clock) : OutputConsumer

data class ConfigError(val displayMessage: String) : ConnectorError

A ConfigError means there is something wrong with the user's connector configuration or any other error for which the connector informs the platform that the error is not transient in nature and thus to not bother retrying.

ConnectorError

sealed interface ConnectorError

Each ConnectorError subtype corresponds to a AirbyteErrorTraceMessage.FailureType.

DefaultExceptionClassifier

@Singleton

@ConfigurationProperties(value = "airbyte.connector.exception-classifiers.default")

class DefaultExceptionClassifier(@Value(value = "${airbyte.connector.exception-classifiers.default.order:1}") val orderValue: Int) : ExceptionClassifier

Default implementation of ExceptionClassifier.

ExceptionClassifier

interface ExceptionClassifier : Ordered

ExceptionHandler

@Singleton

class ExceptionHandler(val classifiers: List<ExceptionClassifier>)

ExceptionHandler applies all available ExceptionClassifier implementations in sequence.

OutputConsumer

@DefaultImplementation(value = StdoutOutputConsumer::class)

abstract class OutputConsumer(clock: Clock) : Consumer<AirbyteMessage> , AutoCloseable

Emits the AirbyteMessage instances produced by the connector.

RecordTemplate

class RecordTemplate(val prefix: ByteArray, val suffix: ByteArray)

RegexExceptionClassifier

@Singleton

@Requires(property = "airbyte.connector.exception-classifiers.regex.rules")

class RegexExceptionClassifier(@Value(value = "${airbyte.connector.exception-classifiers.regex.order:10}") val orderValue: Int, val rules: List<RegexExceptionClassifierRule>) : RuleBasedExceptionClassifier<RegexExceptionClassifierRule>

ExceptionClassifier implementation based on regexes applied to the exception message.

RegexExceptionClassifierRule

@EachProperty(value = "airbyte.connector.exception-classifiers.regex.rules", list = true)

class RegexExceptionClassifierRule(@Parameter val ordinal: Int) : RuleBasedExceptionClassifier.Rule

Micronaut configuration object for RuleBasedExceptionClassifier rules.

RuleBasedExceptionClassifier

interface RuleBasedExceptionClassifier<T : RuleBasedExceptionClassifier.Rule> : ExceptionClassifier

Convenience interface for rules-based ExceptionClassifier implementations.

StandardOutputConsumer

abstract class StandardOutputConsumer(clock: Clock) : BaseStdoutOutputConsumer

A simple OutputConsumer such as standard output or buffering test output consumer.

StdoutOutputConsumer

@Singleton

@Secondary

class StdoutOutputConsumer(val stdout: PrintStream, clock: Clock, @Value(value = "${airbyte.connector.output.buffer-byte-size-threshold-for-flush:4096}") val bufferByteSizeThresholdForFlush: Int) : StandardOutputConsumer

Default implementation of OutputConsumer.

StreamToTemplateMap

typealias StreamToTemplateMap = ConcurrentHashMap<String, RecordTemplate>

SystemError

data class SystemError(val displayMessage: String?) : ConnectorError

A SystemError means there is something wrong with the connector.

TransientError

data class TransientError(val displayMessage: String) : ConnectorError

A TransientError means there is something wrong with the user's source or any other error for which the connector informs the platform that the error is transient in nature.

Properties

CONNECTOR_OUTPUT_PREFIX

const val CONNECTOR_OUTPUT_PREFIX: String

Configuration properties prefix for StdoutOutputConsumer.

DEFAULT_CLASSIFIER_PREFIX

const val DEFAULT_CLASSIFIER_PREFIX: String

EXCEPTION_CLASSIFIER_PREFIX

const val EXCEPTION_CLASSIFIER_PREFIX: String

Common Micronaut property prefix for all exception classifiers.

REGEX_CLASSIFIER_PREFIX

const val REGEX_CLASSIFIER_PREFIX: String