Writes the contents of a FlowFile as an object in a Google Cloud Storage.
google, google cloud, gcs, archive, put
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
Display Name | API Name | Default Value | Allowable Values | Description |
---|---|---|---|---|
Project ID | gcp-project-id | Google Cloud Project ID Supports Expression Language: true (will be evaluated using variable registry only) | ||
GCP Credentials Provider Service | GCP Credentials Provider Service | Controller Service API: GCPCredentialsService Implementation: GCPCredentialsControllerService | The Controller Service used to obtain Google Cloud Platform credentials. | |
Number of retries | gcp-retry-count | 6 | How many retry attempts should be made before routing to the failure relationship. | |
Proxy host | gcp-proxy-host | IP or hostname of the proxy to be used.
You might need to set the following properties in bootstrap for https proxy usage:
-Djdk.http.auth.tunneling.disabledSchemes=
-Djdk.http.auth.proxying.disabledSchemes= Supports Expression Language: true (will be evaluated using variable registry only) | ||
Proxy port | gcp-proxy-port | Proxy port number Supports Expression Language: true (will be evaluated using variable registry only) | ||
HTTP Proxy Username | gcp-proxy-user-name | HTTP Proxy Username Supports Expression Language: true (will be evaluated using variable registry only) | ||
HTTP Proxy Password | gcp-proxy-user-password | HTTP Proxy Password Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only) | ||
Proxy Configuration Service | proxy-configuration-service | Controller Service API: ProxyConfigurationService Implementation: StandardProxyConfigurationService | Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN | |
Storage API URL | storage-api-url | Overrides the default storage URL. Configuring an alternative Storage API URL also overrides the HTTP Host header on requests as described in the Google documentation for Private Service Connections. Supports Expression Language: true (will be evaluated using variable registry only) | ||
Bucket | gcs-bucket | ${gcs.bucket} | Bucket of the object. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | |
Key | gcs-key | ${filename} | Name of the object. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | |
Content Type | gcs-content-type | ${mime.type} | Content Type for the file, i.e. text/plain Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | |
MD5 Hash | gcs-object-md5 | MD5 Hash (encoded in Base64) of the file for server-side validation. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | ||
CRC32C Checksum | gcs-object-crc32c | CRC32C Checksum (encoded in Base64, big-Endian order) of the file for server-side validation. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | ||
Object ACL | gcs-object-acl |
| Access Control to be attached to the object uploaded. Not providing this will revert to bucket defaults. | |
Server Side Encryption Key | gcs-server-side-encryption-key | An AES256 Encryption Key (encoded in base64) for server-side encryption of the object. Sensitive Property: true Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | ||
Overwrite Object | gcs-overwrite-object | true |
| If false, the upload to GCS will succeed only if the object does not exist. |
Content Disposition Type | gcs-content-disposition-type |
| Type of RFC-6266 Content Disposition to be attached to the object | |
GZIP Compression Enabled | gzip.content.enabled | true |
| Signals to the GCS Blob Writer whether GZIP compression during transfer is desired. False means do not gzip and can boost performance in many cases. |
Supports Sensitive Dynamic Properties: No
Dynamic Properties allow the user to specify both the name and value of a property.
Name | Value | Description |
---|---|---|
The name of a User-Defined Metadata field to add to the GCS Object | The value of a User-Defined Metadata field to add to the GCS Object | Allows user-defined metadata to be added to the GCS object as key/value pairs Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) |
Name | Description |
---|---|
success | FlowFiles are routed to this relationship after a successful Google Cloud Storage operation. |
failure | FlowFiles are routed to this relationship if the Google Cloud Storage operation fails. |
Name | Description |
---|---|
filename | Uses the FlowFile's filename as the filename for the GCS object |
mime.type | Uses the FlowFile's MIME type as the content-type for the GCS object |
Name | Description |
---|---|
gcs.bucket | Bucket of the object. |
gcs.key | Name of the object. |
gcs.size | Size of the object. |
gcs.cache.control | Data cache control of the object. |
gcs.component.count | The number of components which make up the object. |
gcs.content.disposition | The data content disposition of the object. |
gcs.content.encoding | The content encoding of the object. |
gcs.content.language | The content language of the object. |
mime.type | The MIME/Content-Type of the object |
gcs.crc32c | The CRC32C checksum of object's data, encoded in base64 in big-endian order. |
gcs.create.time | The creation time of the object (milliseconds) |
gcs.update.time | The last modification time of the object (milliseconds) |
gcs.encryption.algorithm | The algorithm used to encrypt the object. |
gcs.encryption.sha256 | The SHA256 hash of the key used to encrypt the object |
gcs.etag | The HTTP 1.1 Entity tag for the object. |
gcs.generated.id | The service-generated for the object |
gcs.generation | The data generation of the object. |
gcs.md5 | The MD5 hash of the object's data encoded in base64. |
gcs.media.link | The media download link to the object. |
gcs.metageneration | The metageneration of the object. |
gcs.owner | The owner (uploader) of the object. |
gcs.owner.type | The ACL entity type of the uploader of the object. |
gcs.uri | The URI of the object as a string. |