CompressContent

CompressContent 2.1.0

Bundle: org.apache.nifi | nifi-standard-nar
Description: Compresses or decompresses the contents of FlowFiles using a user-specified compression algorithm and updates the mime.type attribute as appropriate. A common idiom is to precede CompressContent with IdentifyMimeType and configure Mode='decompress' AND Compression Format='use mime.type attribute'. When used in this manner, the MIME type is automatically detected and the data is decompressed, if necessary. If decompression is unnecessary, the data is passed through to the 'success' relationship. This processor operates in a very memory efficient way so very large objects well beyond the heap size are generally fine to process.
Tags: brotli, bzip2, compress, content, decompress, deflate, gzip, lz4-framed, lzma, snappy, snappy framed, snappy-hadoop, xz-lzma2, zstd
Input Requirement: REQUIRED
Supports Sensitive Dynamic Properties: false

Properties

Compression Format
The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed
Display Name

Compression Format

Description

The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed

API Name

Compression Format

Default Value

use mime.type attribute

Allowable Values
- use mime.type attribute
- gzip
- deflate
- bzip2
- xz-lzma2
- lzma
- snappy
- snappy-hadoop
- snappy framed
- lz4-framed
- zstd
- brotli
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Compression Level
The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.
Display Name

Compression Level

Description

The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.

API Name

Compression Level

Default Value

1

Allowable Values
- 0
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
Expression Language Scope

Not Supported

Sensitive

false

Required

true

Dependencies
- Compression Format is set to any of [zstd, use mime.type attribute, deflate, brotli, gzip, xz-lzma2]
- Mode is set to any of [compress]
Mode
Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'
Display Name

Mode

Description

Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'

API Name

Mode

Default Value

compress

Allowable Values
- compress
- decompress
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Update Filename
If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data
Display Name

Update Filename

Description

If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data

API Name

Update Filename

Default Value

false

Allowable Values
- true
- false
Expression Language Scope

Not Supported

Sensitive

false

Required

true

System Resource Considerations

Resource	Description
CPU	An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.
MEMORY	An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.

Relationships

Name	Description
success	FlowFiles will be transferred to the success relationship after successfully being compressed or decompressed
failure	FlowFiles will be transferred to the failure relationship if they fail to compress/decompress

Reads Attributes

Name	Description
mime.type	If the Compression Format is set to use mime.type attribute, this attribute is used to determine the compression type. Otherwise, this attribute is ignored.

Writes Attributes

Name	Description
mime.type	If the Mode property is set to compress, the appropriate MIME Type is set. If the Mode property is set to decompress and the file is successfully decompressed, this attribute is removed, as the MIME Type is no longer known.

Use Cases

Compress the contents of a FlowFile

Description

Compress the contents of a FlowFile

Configuration

"Mode" = "compress"
"Compression Format" should be set to whichever compression algorithm should be used.

Decompress the contents of a FlowFile

Description

Decompress the contents of a FlowFile

Configuration

"Mode" = "decompress"
"Compression Format" should be set to whichever compression algorithm was used to compress the data previously.

Use Cases Involving Other Components

Check whether or not a FlowFile is compressed and if so, decompress it.
Description

Check whether or not a FlowFile is compressed and if so, decompress it.

Notes

If IdentifyMimeType determines that the content is not compressed, CompressContent will pass the FlowFile along to the 'success' relationship without attempting to decompress it.

Keywords

auto, detect, mime type, compress, decompress, gzip, bzip2

Processor Configurations
org.apache.nifi.processors.standard.IdentifyMimeType
```
Default property values are sufficient.
Connect the 'success' relationship to CompressContent.
```
org.apache.nifi.processors.standard.CompressContent
```
"Mode" = "decompress"
"Compression Format" = "use mime.type attribute"
```