Connector Datastores
IDOL NiFi Ingest Connectors keep a record of the items they have retrieved so that they do not need to synchronize those items again. Each time it synchronizes with your data repository, a NiFi Ingest Connector only retrieves new items, modified items, and information about the items that have been deleted.
NiFi Ingest Connectors can store this information in an internal datastore, or connect to an external database server. The internal datastore is created automatically by the connector and requires no additional configuration. However, the internal datastore is accessible only to the connector that created it. If you want to run an Apache NiFi cluster, you must configure your NiFi Ingest connectors to store their state information in an external database.
Each NiFi Ingest connector has an optional property named State Database Service. To use an external database, create a database service as described in Create a Database Service, and set this property to the name of the service you create. When you do not set the State Database Service property, the connector uses an internal datastore.
TIP: You can use the same database service for multiple NiFi Ingest connectors. Each connector creates separate tables in the database.
If you modify the configuration of a connector, you might want to delete the state information so that the connector ingests all of the items again. To delete the state information that has been saved by a NiFi Ingest connector, use the following procedure.
To delete state information saved by a connector
- Open the Apache NiFi user interface in your web browser.
- Stop the connector, if it is running. To do this, right-click the processor and click Stop.
-
Right-click the connector and click Configure.
The Configure Processor dialog box opens.
- Click the Properties tab.
-
Click Advanced.
The advanced configuration page opens.
-
Click Purge Datastore.
The state information is deleted.