FetchDistributedMapCache

FetchDistributedMapCache 2.1.0

Bundle: org.apache.nifi | nifi-standard-nar
Description: Computes cache key(s) from FlowFile attributes, for each incoming FlowFile, and fetches the value(s) from the Distributed Map Cache associated with each key. If configured without a destination attribute, the incoming FlowFile's content is replaced with the binary data received by the Distributed Map Cache. If there is no value stored under that key then the flow file will be routed to 'not-found'. Note that the processor will always attempt to read the entire cached value into memory before placing it in it's destination. This could be potentially problematic if the cached value is very large.
Tags: cache, distributed, fetch, map
Input Requirement: REQUIRED
Supports Sensitive Dynamic Properties: false

Properties

Cache Entry Identifier
A comma-delimited list of FlowFile attributes, or the results of Attribute Expression Language statements, which will be evaluated against a FlowFile in order to determine the value(s) used to identify duplicates; it is these values that are cached. NOTE: Only a single Cache Entry Identifier is allowed unless Put Cache Value In Attribute is specified. Multiple cache lookups are only supported when the destination is a set of attributes (see the documentation for 'Put Cache Value In Attribute' for more details including naming convention.

Display Name

Cache Entry Identifier

Description

A comma-delimited list of FlowFile attributes, or the results of Attribute Expression Language statements, which will be evaluated against a FlowFile in order to determine the value(s) used to identify duplicates; it is these values that are cached. NOTE: Only a single Cache Entry Identifier is allowed unless Put Cache Value In Attribute is specified. Multiple cache lookups are only supported when the destination is a set of attributes (see the documentation for 'Put Cache Value In Attribute' for more details including naming convention.

API Name

Cache Entry Identifier

Default Value

${hash.value}

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

true
Character Set
The Character Set in which the cached value is encoded. This will only be used when routing to an attribute.

Display Name

Character Set

Description

The Character Set in which the cached value is encoded. This will only be used when routing to an attribute.

API Name

Character Set

Default Value

UTF-8

Expression Language Scope

Not Supported

Sensitive

false

Required

false
Distributed Cache Service
The Controller Service that is used to get the cached values.

Display Name

Distributed Cache Service

Description

The Controller Service that is used to get the cached values.

API Name

Distributed Cache Service

Service Interface

org.apache.nifi.distributed.cache.client.DistributedMapCacheClient

Service Implementations

org.apache.nifi.hazelcast.services.cacheclient.HazelcastMapCacheClient

org.apache.nifi.distributed.cache.client.MapCacheClientService

org.apache.nifi.redis.service.RedisDistributedMapCacheClientService

org.apache.nifi.redis.service.SimpleRedisDistributedMapCacheClientService

Expression Language Scope

Not Supported

Sensitive

false

Required

true
Max Length To Put In Attribute
If routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.

Display Name

Max Length To Put In Attribute

Description

If routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.

API Name

Max Length To Put In Attribute

Default Value

256

Expression Language Scope

Not Supported

Sensitive

false

Required

false
Put Cache Value In Attribute
If set, the cache value received will be put into an attribute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. If multiple Cache Entry Identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier.

Display Name

Put Cache Value In Attribute

Description

If set, the cache value received will be put into an attribute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. If multiple Cache Entry Identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier.

API Name

Put Cache Value In Attribute

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

false

Relationships

Name	Description
success	If the cache was successfully communicated with it will be routed to this relationship
not-found	If a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationship
failure	If unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationship

Writes Attributes

Name	Description
user-defined	If the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache. If multiple cache entry identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier. For example, if the Cache Entry Identifier property is set to 'id,name', and the user-defined property is named 'fetched', then two attributes will be written, fetched.id and fetched.name, containing their respective values.

Name

Description

user-defined

If the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache. If multiple cache entry identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier. For example, if the Cache Entry Identifier property is set to 'id,name', and the user-defined property is named 'fetched', then two attributes will be written, fetched.id and fetched.name, containing their respective values.