bulk-cdk-toolkit-load-gcs/io.airbyte.cdk.load.file.gcs

Package-level declarations

Types

GcsBlob

data class GcsBlob(val key: String, val storageConfig: GcsClientConfiguration) : RemoteObject<GcsClientConfiguration>

Represents a single blob in Google Cloud Storage.

GcsClient

interface GcsClient : ObjectStorageClient<GcsBlob>

GcsClientFactory

@Factory

class GcsClientFactory(gcsClientConfigurationProvider: GcsClientConfigurationProvider)

GcsNativeClient

class GcsNativeClient(storage: Storage, config: GcsClientConfiguration) : GcsClient

THis client should be used when we finally decide to support native GCP auth. We can then swap over to this client instead of the GcsS3Client which will require us to use HMAC auth.

GcsS3Client

class GcsS3Client(s3Client: S3Client, config: GcsClientConfiguration) : GcsClient

This client is currently the primary class. It is specifically built around the S3Client. It exists to allow us to no load the S3 toolkit and avoid client collision. We specifically need it because we are forced into the HMAC auth for legacy reasons. At the end of the day, this is just a bare-bone wrapper around the S3Client

GcsStreamingUpload

class GcsStreamingUpload(storage: Storage, key: String, config: GcsClientConfiguration, metadata: Map<String, String>) : StreamingUpload<GcsBlob>

This class is responsible for uploading individual parts to GCS and composing them at the end.

Functions

toGcsBlob

fun S3Object.toGcsBlob(config: GcsClientConfiguration): GcsBlob

toS3Object

fun GcsBlob.toS3Object(): S3Object