PackageFlowFile 2.4.0

Bundle
org.apache.nifi | nifi-standard-nar
Description
This processor will package FlowFile attributes and content into an output FlowFile that can be exported from NiFi and imported back into NiFi, preserving the original attributes and content.
Tags
attributes, flowfile, flowfile-stream, flowfile-stream-v3, package
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
Properties
Relationships
Name Description
original The FlowFiles that were used to create the package are sent to this relationship
success The packaged FlowFile is sent to this relationship
Writes Attributes
Name Description
mime.type The mime.type will be changed to application/flowfile-v3
Use Cases Involving Other Components
  • Send FlowFile content and attributes from one NiFi instance to another NiFi instance.
    Description
    Send FlowFile content and attributes from one NiFi instance to another NiFi instance.
    Notes
    A Remote Process Group is preferred to send FlowFiles between two NiFi instances, but an alternative is to use PackageFlowFile then InvokeHTTP sending to ListenHTTP.
    Keywords
    flowfile, attributes, content, ffv3, flowfile-stream-v3, transfer
    Processor Configurations
    org.apache.nifi.processors.standard.PackageFlowFile
        "Maximum Batch Size" > 1 can improve storage or transmission efficiency by batching many FlowFiles together into 1 larger file.
    "Maximum Batch Content Size" can be used to enforce a soft upper limit on the overall package size.

    Note, that the Batch properties only restrict the maximum amount of FlowFiles to incorporate into a single package.
    In case less FlowFiles are queued than the properties allow for,
    the processor will not wait for the limits to be reached but create smaller packages instead.

    Connect the success relationship of PackageFlowFile to the input of InvokeHTTP.
    org.apache.nifi.processors.standard.InvokeHTTP
        "HTTP Method" = "POST" to send data to ListenHTTP.
    "HTTP URL" should include the hostname, port, and path to the ListenHTTP.
    "Request Content-Type" = "${mime.type}" because PackageFlowFile output files have attribute mime.type=application/flowfile-v3.
    org.apache.nifi.processors.standard.ListenHTTP
        "Listening Port" = a unique port number.

    ListenHTTP automatically unpacks files that have attribute mime.type=application/flowfile-v3.
    If PackageFlowFile batches 99 FlowFiles into 1 file that InvokeHTTP sends, then the original 99 FlowFiles will be output by ListenHTTP.
  • Export FlowFile content and attributes from NiFi to external storage and reimport.
    Description
    Export FlowFile content and attributes from NiFi to external storage and reimport.
    Keywords
    flowfile, attributes, content, ffv3, flowfile-stream-v3, offline, storage
    Processor Configurations
    org.apache.nifi.processors.standard.PackageFlowFile
        "Maximum Batch Size" > 1 can improve storage or transmission efficiency by batching many FlowFiles together into 1 larger file.
    "Maximum Batch Content Size" can be used to enforce a soft upper limit on the overall package size.

    Note, that the Batch properties only restrict the maximum amount of FlowFiles to incorporate into a single package.
    In case less FlowFiles are queued than the properties allow for,
    the processor will not wait for the limits to be reached but create smaller packages instead.

    Connect the success relationship to the input of any NiFi egress processor for offline storage.
    org.apache.nifi.processors.standard.UnpackContent
        "Packaging Format" = "application/flowfile-v3".

    Connect the output of a NiFi ingress processor that reads files stored offline to the input of UnpackContent.
    If PackageFlowFile batches 99 FlowFiles into 1 file that is read from storage, then the original 99 FlowFiles will be output by UnpackContent.
See Also