-
Processors
- AttributeRollingWindow
- AttributesToCSV
- AttributesToJSON
- CalculateRecordStats
- CaptureChangeMySQL
- CompressContent
- ConnectWebSocket
- ConsumeAMQP
- ConsumeAzureEventHub
- ConsumeElasticsearch
- ConsumeGCPubSub
- ConsumeIMAP
- ConsumeJMS
- ConsumeKafka
- ConsumeKinesisStream
- ConsumeMQTT
- ConsumePOP3
- ConsumeSlack
- ConsumeTwitter
- ConsumeWindowsEventLog
- ControlRate
- ConvertCharacterSet
- ConvertRecord
- CopyAzureBlobStorage_v12
- CopyS3Object
- CountText
- CryptographicHashContent
- DebugFlow
- DecryptContentAge
- DecryptContentPGP
- DeduplicateRecord
- DeleteAzureBlobStorage_v12
- DeleteAzureDataLakeStorage
- DeleteByQueryElasticsearch
- DeleteDynamoDB
- DeleteFile
- DeleteGCSObject
- DeleteGridFS
- DeleteMongo
- DeleteS3Object
- DeleteSFTP
- DeleteSQS
- DetectDuplicate
- DistributeLoad
- DuplicateFlowFile
- EncodeContent
- EncryptContentAge
- EncryptContentPGP
- EnforceOrder
- EvaluateJsonPath
- EvaluateXPath
- EvaluateXQuery
- ExecuteGroovyScript
- ExecuteProcess
- ExecuteScript
- ExecuteSQL
- ExecuteSQLRecord
- ExecuteStreamCommand
- ExtractAvroMetadata
- ExtractEmailAttachments
- ExtractEmailHeaders
- ExtractGrok
- ExtractHL7Attributes
- ExtractRecordSchema
- ExtractText
- FetchAzureBlobStorage_v12
- FetchAzureDataLakeStorage
- FetchBoxFile
- FetchDistributedMapCache
- FetchDropbox
- FetchFile
- FetchFTP
- FetchGCSObject
- FetchGoogleDrive
- FetchGridFS
- FetchS3Object
- FetchSFTP
- FetchSmb
- FilterAttribute
- FlattenJson
- ForkEnrichment
- ForkRecord
- GenerateFlowFile
- GenerateRecord
- GenerateTableFetch
- GeoEnrichIP
- GeoEnrichIPRecord
- GeohashRecord
- GetAsanaObject
- GetAwsPollyJobStatus
- GetAwsTextractJobStatus
- GetAwsTranscribeJobStatus
- GetAwsTranslateJobStatus
- GetAzureEventHub
- GetAzureQueueStorage_v12
- GetDynamoDB
- GetElasticsearch
- GetFile
- GetFileResource
- GetFTP
- GetGcpVisionAnnotateFilesOperationStatus
- GetGcpVisionAnnotateImagesOperationStatus
- GetHubSpot
- GetMongo
- GetMongoRecord
- GetS3ObjectMetadata
- GetSFTP
- GetShopify
- GetSmbFile
- GetSNMP
- GetSplunk
- GetSQS
- GetWorkdayReport
- GetZendesk
- HandleHttpRequest
- HandleHttpResponse
- IdentifyMimeType
- InvokeHTTP
- InvokeScriptedProcessor
- ISPEnrichIP
- JoinEnrichment
- JoltTransformJSON
- JoltTransformRecord
- JSLTTransformJSON
- JsonQueryElasticsearch
- ListAzureBlobStorage_v12
- ListAzureDataLakeStorage
- ListBoxFile
- ListDatabaseTables
- ListDropbox
- ListenFTP
- ListenHTTP
- ListenOTLP
- ListenSlack
- ListenSyslog
- ListenTCP
- ListenTrapSNMP
- ListenUDP
- ListenUDPRecord
- ListenWebSocket
- ListFile
- ListFTP
- ListGCSBucket
- ListGoogleDrive
- ListS3
- ListSFTP
- ListSmb
- LogAttribute
- LogMessage
- LookupAttribute
- LookupRecord
- MergeContent
- MergeRecord
- ModifyBytes
- ModifyCompression
- MonitorActivity
- MoveAzureDataLakeStorage
- Notify
- PackageFlowFile
- PaginatedJsonQueryElasticsearch
- ParseEvtx
- ParseNetflowv5
- ParseSyslog
- ParseSyslog5424
- PartitionRecord
- PublishAMQP
- PublishGCPubSub
- PublishJMS
- PublishKafka
- PublishMQTT
- PublishSlack
- PutAzureBlobStorage_v12
- PutAzureCosmosDBRecord
- PutAzureDataExplorer
- PutAzureDataLakeStorage
- PutAzureEventHub
- PutAzureQueueStorage_v12
- PutBigQuery
- PutBoxFile
- PutCloudWatchMetric
- PutDatabaseRecord
- PutDistributedMapCache
- PutDropbox
- PutDynamoDB
- PutDynamoDBRecord
- PutElasticsearchJson
- PutElasticsearchRecord
- PutEmail
- PutFile
- PutFTP
- PutGCSObject
- PutGoogleDrive
- PutGridFS
- PutKinesisFirehose
- PutKinesisStream
- PutLambda
- PutMongo
- PutMongoBulkOperations
- PutMongoRecord
- PutRecord
- PutRedisHashRecord
- PutS3Object
- PutSalesforceObject
- PutSFTP
- PutSmbFile
- PutSNS
- PutSplunk
- PutSplunkHTTP
- PutSQL
- PutSQS
- PutSyslog
- PutTCP
- PutUDP
- PutWebSocket
- PutZendeskTicket
- QueryAirtableTable
- QueryAzureDataExplorer
- QueryDatabaseTable
- QueryDatabaseTableRecord
- QueryRecord
- QuerySalesforceObject
- QuerySplunkIndexingStatus
- RemoveRecordField
- RenameRecordField
- ReplaceText
- ReplaceTextWithMapping
- RetryFlowFile
- RouteHL7
- RouteOnAttribute
- RouteOnContent
- RouteText
- RunMongoAggregation
- SampleRecord
- ScanAttribute
- ScanContent
- ScriptedFilterRecord
- ScriptedPartitionRecord
- ScriptedTransformRecord
- ScriptedValidateRecord
- SearchElasticsearch
- SegmentContent
- SendTrapSNMP
- SetSNMP
- SignContentPGP
- SplitAvro
- SplitContent
- SplitExcel
- SplitJson
- SplitPCAP
- SplitRecord
- SplitText
- SplitXml
- StartAwsPollyJob
- StartAwsTextractJob
- StartAwsTranscribeJob
- StartAwsTranslateJob
- StartGcpVisionAnnotateFilesOperation
- StartGcpVisionAnnotateImagesOperation
- TagS3Object
- TailFile
- TransformXml
- UnpackContent
- UpdateAttribute
- UpdateByQueryElasticsearch
- UpdateCounter
- UpdateDatabaseTable
- UpdateRecord
- ValidateCsv
- ValidateJson
- ValidateRecord
- ValidateXml
- VerifyContentMAC
- VerifyContentPGP
- Wait
-
Controller Services
- ADLSCredentialsControllerService
- ADLSCredentialsControllerServiceLookup
- AmazonGlueSchemaRegistry
- ApicurioSchemaRegistry
- AvroReader
- AvroRecordSetWriter
- AvroSchemaRegistry
- AWSCredentialsProviderControllerService
- AzureBlobStorageFileResourceService
- AzureCosmosDBClientService
- AzureDataLakeStorageFileResourceService
- AzureEventHubRecordSink
- AzureStorageCredentialsControllerService_v12
- AzureStorageCredentialsControllerServiceLookup_v12
- CEFReader
- ConfluentEncodedSchemaReferenceReader
- ConfluentEncodedSchemaReferenceWriter
- ConfluentSchemaRegistry
- CSVReader
- CSVRecordLookupService
- CSVRecordSetWriter
- DatabaseRecordLookupService
- DatabaseRecordSink
- DatabaseTableSchemaRegistry
- DBCPConnectionPool
- DBCPConnectionPoolLookup
- DistributedMapCacheLookupService
- ElasticSearchClientServiceImpl
- ElasticSearchLookupService
- ElasticSearchStringLookupService
- EmailRecordSink
- EmbeddedHazelcastCacheManager
- ExcelReader
- ExternalHazelcastCacheManager
- FreeFormTextRecordSetWriter
- GCPCredentialsControllerService
- GCSFileResourceService
- GrokReader
- HazelcastMapCacheClient
- HikariCPConnectionPool
- HttpRecordSink
- IPLookupService
- JettyWebSocketClient
- JettyWebSocketServer
- JMSConnectionFactoryProvider
- JndiJmsConnectionFactoryProvider
- JsonConfigBasedBoxClientService
- JsonPathReader
- JsonRecordSetWriter
- JsonTreeReader
- Kafka3ConnectionService
- KerberosKeytabUserService
- KerberosPasswordUserService
- KerberosTicketCacheUserService
- LoggingRecordSink
- MapCacheClientService
- MapCacheServer
- MongoDBControllerService
- MongoDBLookupService
- PEMEncodedSSLContextProvider
- PropertiesFileLookupService
- ProtobufReader
- ReaderLookup
- RecordSetWriterLookup
- RecordSinkServiceLookup
- RedisConnectionPoolService
- RedisDistributedMapCacheClientService
- RestLookupService
- S3FileResourceService
- ScriptedLookupService
- ScriptedReader
- ScriptedRecordSetWriter
- ScriptedRecordSink
- SetCacheClientService
- SetCacheServer
- SimpleCsvFileLookupService
- SimpleDatabaseLookupService
- SimpleKeyValueLookupService
- SimpleRedisDistributedMapCacheClientService
- SimpleScriptedLookupService
- SiteToSiteReportingRecordSink
- SlackRecordSink
- SmbjClientProviderService
- StandardAsanaClientProviderService
- StandardAzureCredentialsControllerService
- StandardDropboxCredentialService
- StandardFileResourceService
- StandardHashiCorpVaultClientService
- StandardHttpContextMap
- StandardJsonSchemaRegistry
- StandardKustoIngestService
- StandardKustoQueryService
- StandardOauth2AccessTokenProvider
- StandardPGPPrivateKeyService
- StandardPGPPublicKeyService
- StandardPrivateKeyService
- StandardProxyConfigurationService
- StandardRestrictedSSLContextService
- StandardS3EncryptionService
- StandardSSLContextService
- StandardWebClientServiceProvider
- Syslog5424Reader
- SyslogReader
- UDPEventRecordSink
- VolatileSchemaCache
- WindowsEventLogReader
- XMLFileLookupService
- XMLReader
- XMLRecordSetWriter
- YamlTreeReader
- ZendeskRecordSink
FetchFTP 2.1.0
- Bundle
- org.apache.nifi | nifi-standard-nar
- Description
- Fetches the content of a file from a remote FTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.
- Tags
- fetch, files, ftp, get, ingest, input, remote, retrieve, source
- Input Requirement
- REQUIRED
- Supports Sensitive Dynamic Properties
- false
Properties
-
Completion Strategy
Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.
- Display Name
- Completion Strategy
- Description
- Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.
- API Name
- Completion Strategy
- Default Value
- None
- Allowable Values
-
- None
- Move File
- Delete File
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Connection Mode
The FTP Connection Mode
- Display Name
- Connection Mode
- Description
- The FTP Connection Mode
- API Name
- Connection Mode
- Default Value
- Passive
- Allowable Values
-
- Active
- Passive
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- false
-
Connection Timeout
Amount of time to wait before timing out while creating a connection
- Display Name
- Connection Timeout
- Description
- Amount of time to wait before timing out while creating a connection
- API Name
- Connection Timeout
- Default Value
- 30 sec
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Create Directory
Used when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.
- Display Name
- Create Directory
- Description
- Used when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.
- API Name
- Create Directory
- Default Value
- false
- Allowable Values
-
- true
- false
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- false
-
Data Timeout
When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
- Display Name
- Data Timeout
- Description
- When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
- API Name
- Data Timeout
- Default Value
- 30 sec
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Log level when file not found
Log level to use in case the file does not exist when the processor is triggered
- Display Name
- Log level when file not found
- Description
- Log level to use in case the file does not exist when the processor is triggered
- API Name
- fetchfiletransfer-notfound-loglevel
- Default Value
- ERROR
- Allowable Values
-
- TRACE
- DEBUG
- INFO
- WARN
- ERROR
- FATAL
- NONE
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Use UTF-8 Encoding
Tells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.
- Display Name
- Use UTF-8 Encoding
- Description
- Tells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.
- API Name
- ftp-use-utf8
- Default Value
- false
- Allowable Values
-
- true
- false
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Hostname
The fully-qualified hostname or IP address of the host to fetch the data from
- Display Name
- Hostname
- Description
- The fully-qualified hostname or IP address of the host to fetch the data from
- API Name
- Hostname
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- false
- Required
- true
-
Internal Buffer Size
Set the internal buffer size for buffered data streams
- Display Name
- Internal Buffer Size
- Description
- Set the internal buffer size for buffered data streams
- API Name
- Internal Buffer Size
- Default Value
- 16KB
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- false
-
Move Destination Directory
The directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.
- Display Name
- Move Destination Directory
- Description
- The directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.
- API Name
- Move Destination Directory
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- false
- Required
- false
-
Password
Password for the user account
- Display Name
- Password
- Description
- Password for the user account
- API Name
- Password
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- true
- Required
- false
-
Port
The port to connect to on the remote host to fetch the data from
- Display Name
- Port
- Description
- The port to connect to on the remote host to fetch the data from
- API Name
- Port
- Default Value
- 21
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- false
- Required
- true
-
Proxy Configuration Service
Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN, SOCKS + AuthN
- Display Name
- Proxy Configuration Service
- Description
- Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN, SOCKS + AuthN
- API Name
- proxy-configuration-service
- Service Interface
- org.apache.nifi.proxy.ProxyConfigurationService
- Service Implementations
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- false
-
Remote File
The fully qualified filename on the remote system
- Display Name
- Remote File
- Description
- The fully qualified filename on the remote system
- API Name
- Remote File
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- false
- Required
- true
-
Transfer Mode
The FTP Transfer Mode
- Display Name
- Transfer Mode
- Description
- The FTP Transfer Mode
- API Name
- Transfer Mode
- Default Value
- Binary
- Allowable Values
-
- Binary
- ASCII
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- false
-
Use Compression
Indicates whether or not ZLIB compression should be used when transferring files
- Display Name
- Use Compression
- Description
- Indicates whether or not ZLIB compression should be used when transferring files
- API Name
- Use Compression
- Default Value
- false
- Allowable Values
-
- true
- false
- Expression Language Scope
- Not Supported
- Sensitive
- false
- Required
- true
-
Username
Username
- Display Name
- Username
- Description
- Username
- API Name
- Username
- Expression Language Scope
- Environment variables and FlowFile Attributes
- Sensitive
- false
- Required
- true
Relationships
Name | Description |
---|---|
success | All FlowFiles that are received are routed to success |
comms.failure | Any FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship. |
not.found | Any FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship. |
permission.denied | Any FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship. |
Writes Attributes
Name | Description |
---|---|
ftp.remote.host | The hostname or IP address from which the file was pulled |
ftp.remote.port | The port that was used to communicate with the remote FTP server |
ftp.remote.filename | The name of the remote file that was pulled |
filename | The filename is updated to point to the filename fo the remote file |
path | If the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute |
fetch.failure.reason | The name of the failure relationship applied when routing to any failure relationship |
Use Cases Involving Other Components
-
Retrieve all files in a directory of an FTP Server
- Description
- Retrieve all files in a directory of an FTP Server
- Keywords
- ftp, file, transform, state, retrieve, fetch, all, stream
- Processor Configurations
-
org.apache.nifi.processors.standard.ListFTP
The "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{FTP_SERVER}`. The "Remote Path" property must be set to the directory on the FTP Server where the files reside. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{FTP_REMOTE_PATH}`. Configure the "Username" property to the appropriate username for logging into the FTP Server. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_USERNAME}`. Configure the "Password" property to the appropriate password for the provided username. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_PASSWORD}`. The 'success' Relationship of this Processor is then connected to FetchFTP.
org.apache.nifi.processors.standard.FetchFTP"Hostname" = "${ftp.remote.host}" "Remote File" = "${path}/${filename}" "Username" = "${ftp.listing.user}" "Password" = "#{FTP_PASSWORD}"
See Also