Manually Generate Document Flow Files
After constructing your ingestion pipeline you might want to test it by sending a small number of documents through the data flow. IDOL NiFi Ingest includes a processor, named GenerateDocumentFlowFile, that you can use to manually create FlowFiles that represent documents.
To generate document FlowFiles
- Add the GenerateDocumentFlowFile processor to the canvas.
- Right-click the GenerateDocumentFlowFile processor and click Configure.
-
Click Advanced.
The Advanced configuration interface opens. This interface allows you to manually generate FlowFiles that represent documents.
-
Create a document FlowFile:
-
In the Documents area, click
.
A dialog box opens.
-
Type a reference for the document, and the name of the outgoing relationship to send the FlowFile to. If the outgoing relationship does not exist it will be created automatically. Then, click ADD.
The new document is added to the Documents list.
- In the XML Metadata area, specify any document metadata that you want to add to the document.
- In the Attributes area, specify any attributes that you want to add to the FlowFile.
-
In the Parts area, specify the document content. You can add the following types of part:
Content part Type plain text to use as the document content, or specify the path of a text file that contains the text. Filename part Type the path of a file (located on the Apache NiFi host machine) to use as the document content. File part Upload a file through the web browser. - Click SAVE.
-
- (Optional) Create further document FlowFiles. You can also delete documents you have created - click the remove icon (
) for the relevant document in the Documents list.
- Close the advanced configuration interface.
- Return to the canvas and connect the output relationships of the GenerateDocumentFlowFile processor to the start of your ingestion pipeline. By default the only output relationship is "success".
-
Start the GenerateDocumentFlowFile processor.
Each time you start the processor, the documents that you created are sent to the relevant output relationship(s).
TIP: To see how documents change as they move through the ingestion pipeline you can stop a processor and inspect the documents in the preceding queue (see Download Documents from a Queue) or insert a WriteDocument processor in the data flow to write the documents to disk.