BatchCallback

Sends callback HTTP requests to your application, to let it know that processing steps have been completed for documents in your NiFi dataflow.

For example, if you create a dataflow to handle connector actions such as collect, you can route the FlowFiles produced by the connector through a series of processing steps, finishing with a batch callback processor. When it receives the FlowFiles, the batch callback processor reports back to your application that processing is complete.

TIP: You can use the QueueInfo action to determine whether an action completed successfully, but this does not take into account any subsequent processing.

Properties

Name Default Value Description
Batch Size 100 The maximum number of FlowFiles included in a single callback.
Batch Interval 60 sec The maximum amount of time to wait for incoming FlowFiles. If the time limit is reached the processor sends the callback regardless of the number of FlowFiles.
Output Format XML Specifies whether the information in the body of the callback is in XML or JSON format.
Grouping Attributes   A comma-separated list of FlowFile attributes to use to group FlowFiles. When you set this property, each callback only includes FlowFiles where all of the attributes are equal. Otherwise, any FlowFiles can be included in the same batch.
Callback URL  

The URL to send the callback to. The processor sends an HTTP POST request with Content-Type either text/xml or application/json.

You can use expression language that references variable registries or Grouping Attributes. For example:

http://localhost:10001/?batch_id=${idol.aci.param.batch_id}

Items Name items The name of the XML root element or JSON root object property in the body of the callback.
Item Name item (XML output format only) The name of the XML element that represents an input FlowFile.
Items Field Names  

These properties map a comma-separated list of FlowFile attributes (Items Field Attributes) to a comma-separated list of XML elements or JSON properties in the output (Items Field Names).

The information is added to the "items" XML element or the root JSON object. You can only specify FlowFile attributes that you set in the property Grouping Attributes.

Items Field Attributes  
Item Field Names reference, identifier, exception

These properties map a comma-separated list of FlowFile attributes (Item Field Attributes) to a comma-separated list of XML elements or JSON properties in the output (Item Field Names).

The information is added to each "item" XML element or to each item in the JSON array of items.

Item Field Attributes idol.reference, idol.doc.identifier, idol.exception
Proxy Host   The host name or IP address of the proxy server to use for sending the callback.
Proxy Port   The port of the proxy server to use for sending the callback.
Dynamic properties   You can set standard IDOL HTTP client options (for example BasicUsername and BasicPassword) as dynamic properties.

Relationships

Name Description
batch FlowFiles representing callbacks are routed to this relationship. If you configure a Callback URL, the information is sent to the URL and a copy is output to this relationship.
success All incoming FlowFiles are routed to this relationship.

Example Output

The following is example output from the batch callback processor, in XML format.

<items>
    <connectorgroup>filesystem</connectorgroup>
    <item>
        <identifier>AUTN_IDENTIFIER_BASE64</identifier>
        <reference>/home/user/files/file1.txt</reference>
    </item>
    <item>
        <identifier>AUTN_IDENTIFIER_BASE64</identifier>
        <reference>/home/user/files/file2.txt</reference>
    </item>
    ...
</items>

To produce this output, you could use the following configuration.

Property Name Value
Output Format XML
Grouping Attributes idol.connector.group
Items Name items
Item Name item
Items Field Names connectorgroup
Items Field Attributes idol.connector.group
Item Field Names reference, identifier, exception
Item Field Attributes idol.reference, idol.doc.identifier, idol.exception