VariableContext

inner class VariableContext(val stream: DestinationStream, val syncTime: Instant = Instant.ofEpochMilli(timeProvider.syncTimeMillis()), val currentTimeProvider: TimeProvider = timeProvider, val extension: String? = null, val partNumber: Long? = null)

Variable substitution is complex.

  1. There are two types: path variables and file name variables.

  2. Path variables use the ${NAME} syntax, while file name variables use the {name} syntax. (I have no idea why this is.)

  3. A variable is defined by a Variable.pattern and a Variable.provider

  4. Variable.provider is a function that takes a VariableContext and returns a string. It's used for substitution to get the actual path.

  5. Variable.pattern is a regex pattern that can match any results of Variable.provider.

  6. If Variable.pattern is null, Variable.provider is used to get the value. (Ie, we won't match against a pattern, but always against the realized value. In practice this is for stream name and namespace, because matching always performed at the stream level.)

  7. Matching should be considered deprecated. It is only required for configurations that do not enable staging, which populate destination state by collecting metadata from object headers. It is extremely brittle and can break against malformed paths or paths that do not include enough variables to avoid clashes. If you run into a client issue which requires a path change anyway (a breaking change for some workflows), consider advising them to enable staging.

Constructors

Link copied to clipboard
constructor(stream: DestinationStream, syncTime: Instant = Instant.ofEpochMilli(timeProvider.syncTimeMillis()), currentTimeProvider: TimeProvider = timeProvider, extension: String? = null, partNumber: Long? = null)

Properties

Link copied to clipboard
Link copied to clipboard
val extension: String? = null
Link copied to clipboard
val partNumber: Long? = null
Link copied to clipboard
Link copied to clipboard