RouteText

RouteText 2.0.0

Bundle: org.apache.nifi | nifi-standard-nar
Description: Routes textual data based on a set of user-defined rules. Each line in an incoming FlowFile is compared against the values specified by user-defined Properties. The mechanism by which the text is compared to these user-defined properties is defined by the 'Matching Strategy'. The data is then routed according to these rules, routing each line of the text individually.
Tags: Expression Language, Regular Expression, attributes, csv, delimited, detect, filter, find, logs, regex, regexp, routing, search, string, text
Input Requirement: REQUIRED
Supports Sensitive Dynamic Properties: false

Properties

Character Set
The Character Set in which the incoming text is encoded

Display Name

Character Set

Description

The Character Set in which the incoming text is encoded

API Name

Character Set

Default Value

UTF-8

Expression Language Scope

Not Supported

Sensitive

false

Required

true
Grouping Regular Expression
Specifies a Regular Expression to evaluate against each line to determine which Group the line should be placed in. The Regular Expression must have at least one Capturing Group that defines the line's Group. If multiple Capturing Groups exist in the Regular Expression, the values from all Capturing Groups will be concatenated together. Two lines will not be placed into the same FlowFile unless they both have the same value for the Group (or neither line matches the Regular Expression). For example, to group together all lines in a CSV File by the first column, we can set this value to "(.*?),.*". Two lines that have the same Group but different Relationships will never be placed into the same FlowFile.

Display Name

Grouping Regular Expression

Description

Specifies a Regular Expression to evaluate against each line to determine which Group the line should be placed in. The Regular Expression must have at least one Capturing Group that defines the line's Group. If multiple Capturing Groups exist in the Regular Expression, the values from all Capturing Groups will be concatenated together. Two lines will not be placed into the same FlowFile unless they both have the same value for the Group (or neither line matches the Regular Expression). For example, to group together all lines in a CSV File by the first column, we can set this value to "(.*?),.*". Two lines that have the same Group but different Relationships will never be placed into the same FlowFile.

API Name

Grouping Regular Expression

Expression Language Scope

Not Supported

Sensitive

false

Required

false
Ignore Case
If true, capitalization will not be taken into account when comparing values. E.g., matching against 'HELLO' or 'hello' will have the same result. This property is ignored if the 'Matching Strategy' is set to 'Satisfies Expression'.
Display Name

Ignore Case

Description

If true, capitalization will not be taken into account when comparing values. E.g., matching against 'HELLO' or 'hello' will have the same result. This property is ignored if the 'Matching Strategy' is set to 'Satisfies Expression'.

API Name

Ignore Case

Default Value

false

Allowable Values
- true
- false
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Ignore Leading/Trailing Whitespace
Indicates whether or not the whitespace at the beginning and end of the lines should be ignored when evaluating the line.

Display Name

Ignore Leading/Trailing Whitespace

Description

Indicates whether or not the whitespace at the beginning and end of the lines should be ignored when evaluating the line.

API Name

Ignore Leading/Trailing Whitespace

Default Value

true

Expression Language Scope

Not Supported

Sensitive

false

Required

true
Matching Strategy
Specifies how to evaluate each line of incoming text against the user-defined properties.
Display Name

Matching Strategy

Description

Specifies how to evaluate each line of incoming text against the user-defined properties.

API Name

Matching Strategy

Allowable Values
- Satisfies Expression
- Starts With
- Ends With
- Contains
- Equals
- Matches Regular Expression
- Contains Regular Expression
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Routing Strategy
Specifies how to determine which Relationship(s) to use when evaluating the lines of incoming text against the 'Matching Strategy' and user-defined properties.
Display Name

Routing Strategy

Description

Specifies how to determine which Relationship(s) to use when evaluating the lines of incoming text against the 'Matching Strategy' and user-defined properties.

API Name

Routing Strategy

Default Value

Route to each matching Property Name

Allowable Values
- Route to each matching Property Name
- Route to 'matched' if line matches all conditions
- Route to 'matched' if lines matches any condition
Expression Language Scope

Not Supported

Sensitive

false

Required

true

Dynamic Properties

Relationship Name
Routes data that matches the value specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property Key.

Name

Relationship Name

Description

Routes data that matches the value specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property Key.

Value

value to match against

Expression Language Scope

FLOWFILE_ATTRIBUTES

Relationships

Name	Description
unmatched	Data that does not satisfy the required user-defined rules will be routed to this Relationship
original	The original input file will be routed to this destination when the lines have been successfully routed to 1 or more relationships

Writes Attributes

Name	Description
RouteText.Route	The name of the relationship to which the FlowFile was routed.
RouteText.Group	The value captured by all capturing groups in the 'Grouping Regular Expression' property. If this property is not set or contains no capturing groups, this attribute will not be added.

Use Cases

Drop blank or empty lines from the FlowFile's content.

Description

Drop blank or empty lines from the FlowFile's content.

Keywords

filter, drop, empty, blank, remove, delete, strip out, lines, text

Configuration

"Routing Strategy" = "Route to each matching Property Name"
"Matching Strategy" = "Matches Regular Expression"
"Empty Line" = "^$"

Auto-terminate the "Empty Line" relationship.
Connect the "unmatched" relationship to the next processor in your flow.

Remove specific lines of text from a file, such as those containing a specific word or having a line length over some threshold.

Description

Remove specific lines of text from a file, such as those containing a specific word or having a line length over some threshold.

Keywords

filter, drop, empty, blank, remove, delete, strip out, lines, text, expression language

Configuration

"Routing Strategy" = "Route to each matching Property Name"
"Matching Strategy" = "Satisfies Expression"

An additional property should be added named "Filter Out." The value should be a NiFi Expression Language Expression that can refer to two variables (in addition to FlowFile attributes): `line`, which is the line of text being evaluated; and `lineNo`, which is the line number in the file (starting with 1). The Expression should return `true` for any line that should be dropped.

For example, to remove any line that starts with a # symbol, we can set "Filter Out" to `${line:startsWith("#")}`.
We could also remove the first 2 lines of text by setting "Filter Out" to `${lineNo:le(2)}`. Note that we use the `le` function because we want lines numbers less than or equal to `2`, since the line index is 1-based.

Auto-terminate the "Filter Out" relationship.
Connect the "unmatched" relationship to the next processor in your flow.