BatchCallback
Sends callback HTTP requests to your application, to let it know that processing steps have been completed for documents in your NiFi dataflow.
For example, if you create a dataflow to handle connector actions such as collect, you can route the FlowFiles produced by the connector through a series of processing steps, finishing with a batch callback processor. When it receives the FlowFiles, the batch callback processor reports back to your application that processing is complete.
TIP: You can use the QueueInfo action to determine whether an action completed successfully, but this does not take into account any subsequent processing.
Properties
Name | Default Value | Description |
---|---|---|
Batch Size | 100 | The maximum number of FlowFiles included in a single callback. |
Batch Interval | 60 sec | The maximum amount of time to wait for incoming FlowFiles. If the time limit is reached the processor sends the callback regardless of the number of FlowFiles. |
Output Format | XML | Specifies whether the information in the body of the callback is in XML or JSON format. |
Grouping Attributes | A comma-separated list of FlowFile attributes to use to group FlowFiles. When you set this property, each callback only includes FlowFiles where all of the attributes are equal. Otherwise, any FlowFiles can be included in the same batch. | |
Callback URL |
The URL to send the callback to. The processor sends an HTTP POST request with Content-Type either You can use expression language that references variable registries or Grouping Attributes. For example:
|
|
Items Name | items | The name of the XML root element or JSON root object property in the body of the callback. |
Item Name | item | (XML output format only) The name of the XML element that represents an input FlowFile. |
Items Field Names |
These properties map a comma-separated list of FlowFile attributes (Items Field Attributes) to a comma-separated list of XML elements or JSON properties in the output (Items Field Names). The information is added to the "items" XML element or the root JSON object. You can only specify FlowFile attributes that you set in the property Grouping Attributes. |
|
Items Field Attributes | ||
Item Field Names | reference, identifier, exception |
These properties map a comma-separated list of FlowFile attributes (Item Field Attributes) to a comma-separated list of XML elements or JSON properties in the output (Item Field Names). The information is added to each "item" XML element or to each item in the JSON array of items. |
Item Field Attributes | idol.reference, idol.doc.identifier, idol.exception | |
Proxy Host | The host name or IP address of the proxy server to use for sending the callback. | |
Proxy Port | The port of the proxy server to use for sending the callback. | |
Dynamic properties | You can set standard IDOL HTTP client options (for example BasicUsername and BasicPassword ) as dynamic properties. |
Relationships
Name | Description |
---|---|
batch | FlowFiles representing callbacks are routed to this relationship. If you configure a Callback URL, the information is sent to the URL and a copy is output to this relationship. |
success | All incoming FlowFiles are routed to this relationship. |
Example Output
The following is example output from the batch callback processor, in XML format.
<items> <connectorgroup>filesystem</connectorgroup> <item> <identifier>AUTN_IDENTIFIER_BASE64</identifier> <reference>/home/user/files/file1.txt</reference> </item> <item> <identifier>AUTN_IDENTIFIER_BASE64</identifier> <reference>/home/user/files/file2.txt</reference> </item> ... </items>
To produce this output, you could use the following configuration.
Property Name | Value |
---|---|
Output Format | XML |
Grouping Attributes | idol.connector.group
|
Items Name | items
|
Item Name | item
|
Items Field Names | connectorgroup
|
Items Field Attributes | idol.connector.group
|
Item Field Names | reference, identifier, exception
|
Item Field Attributes | idol.reference, idol.doc.identifier, idol.exception
|