QueryIDOL

A processor that sends a query to an IDOL Content component and outputs the resulting documents as FlowFiles.

You can specify a query in one of the following ways:

  • Define a query using the Text, Field Text, Database Match, and other relevant properties in the processor configuration. With this approach you can define a single, pre-configured query and there is no incoming connection. The processor sends a Query action to IDOL Content each time it is scheduled to run, so Micro Focus recommends that you check the scheduling settings and configure a reasonable interval between each cycle. You can choose whether the second and subsequent queries return all matching documents, or only new documents, by setting the property New documents only.
  • Initiate a query by routing a FlowFile to the processor. The query parameters (including text, field text, and database name) can be set as FlowFile attributes. In this case, configure the processor properties using NiFi expression language that references the FlowFile attributes. The incoming FlowFile is routed to the "original" relationship, with additional attributes that you can use to construct a new query to retrieve new documents (the same query performed later could return additional results, if new documents are indexed). In this case the scheduling options define how often the processor checks its incoming connection for queued FlowFiles.

This processor can be useful when you want to re-ingest some of the documents in your IDOL index. You can send a query to IDOL Content and then route the result documents back to an IDOL Connector so that the connector retrieves those documents again.

NOTE: The connector must support the synchronize from identifiers feature.

NOTE: Your query results might contain documents retrieved by many different connectors. Each document must be routed back to the correct connector. You can route the resulting FlowFiles by the value of the idol.doc.source attribute (for a description of this attribute, see Introduction to FlowFiles and Documents). Apache NiFi includes a built-in processor named RouteOnAttribute that can route FlowFiles based on attribute values.

Properties

Name Default Value Description
IDOL License Service  

An IdolLicenseServiceImpl that provides a way to communicate with an IDOL License Server.

IDOL Host   The host name or IP address of the IDOL Content component.
IDOL ACI Port   The ACI port of the IDOL Content component.
SSL Config Service   An optional IdolSSLConfigServiceImpl that specifies the settings to use to communicate over SSL/TLS. Set this property only if your IDOL Content component has been configured to accept connections over SSL.
Batch Size 100 The number of documents to release to the output relationship at one time.
Text   Sets the value of the Text parameter in the Query action.
Field Text   Sets the value of the FieldText parameter in the Query action.
Database Match   Sets the value of the DatabaseMatch parameter in the Query action.
Print AllSections Sets the value of the Print parameter in the Query action.
Print Fields AUTN_SOURCE, AUTN_IDENTIFIER Sets the value of the PrintFields parameter in the Query action.
New documents only True

Specifies whether to return only new documents.

  • When you set this property to FALSE, the processor outputs a FlowFile for every document in the IDOL index that matches the query. (With an incoming connection, this instructs the processor to ignore the idol.query.result.lastdocid attribute in incoming FlowFiles - see the description of the original relationship, below).
  • When you set this property to TRUE, the processor outputs a FlowFile for every document in the IDOL index that matches the query, but ignores documents that were returned on previous cycles. (With an incoming connection, requires you to set the attribute idol.query.result.lastdocid on incoming FlowFiles).
XML reference path DREREFERENCE The field, in documents in the index, that contains the document reference.
XML source path AUTN_SOURCE The field, in documents in the index, that contains the document source.
XML identifier path AUTN_IDENTIFIER The field, in documents in the index, that contains the document identifier.
XML content path DRECONTENT The field, in documents in the index, that contains the document content.
Dynamic properties  

Any dynamic properties that you set are added to the query action as action parameters. For information about the action parameters that you can use, refer to the IDOL Content Component Reference.

NOTE: You cannot set the action parameters Combine, StoreState, StateMatchID, MinID, or MaxResults because these are set automatically.

attr:name  

When you configure the processor to run queries in response to incoming FlowFiles, you might want to add information about a query to all of the results.

To do this, add dynamic properties named attr:name, where name is the name of a FlowFile attribute to add to each result FlowFile. The value of the dynamic property sets the value, and can be a constant or can use NiFi expression language to copy a FlowFile attribute from the incoming FlowFile.

Relationships

Name Description
success FlowFiles that represent documents returned from the query action that was sent to the IDOL Content component.
original

This relationship is used only when the processor has an incoming connection. Incoming FlowFiles that initiate a query are routed to this relationship.

The processor adds the following attributes to each processed FlowFile:

  • idol.query.result.lastdocid - the largest document ID returned by the query. You can set this attribute in FlowFiles that you route to the QueryIDOL processor, to retrieve only new documents (the same query performed later could return additional results, if new documents are indexed).
  • idol.query.result.hitcount - the total number of documents routed to the success relationship.

View and Clear State Information

When there is no incoming connection, the processor stores state information so that it can obtain only new documents. If you change the query that you have configured, Micro Focus recommends clearing the state information.

To clear the state information

  1. Stop the processor.
  2. Right-click the processor and click View State.

    A dialog box opens.

  3. Click Clear state and then click CLOSE.

Stored State Tokens

When a query is complete the processor uses the TokenManagement action to delete the state token in IDOL Content. This is an administrative action and fails unless NiFi is authorized to use it. There is no effect on NiFi Ingest if the TokenManagement action fails, but the state token is not deleted.