DistributeOnBatch
Set DistributeOnBatch
to True
if you want the DIH to index the documents it receives to alternating child IDOL servers. You can use this distribution mode only when MirrorMode is set to False
.
By default, in non-mirror mode DIH distributes documents evenly to all child servers. It sends each incoming IDX or XML file to all child servers, so that the child server can use the documents for deduplication. However, each child server indexes only a proportion of the documents in each IDX file.
In DistributeOnBatch
mode, DIH sends each IDX or XML file to only one child server (or mirrored server group). This child server indexes all the documents in the IDX or XML file. DIH rotates the child server that it sends IDX files to, so that the documents are evenly distributed between all child servers.
This mode reduces the volume of data sent between DIH and its child servers, which can improve indexing efficiency.
NOTE: You can use DistributeOnBatch
mode only if you do not require deduplication of documents. Because DIH rotates the child server that receives an IDX file, two copies of a duplicate document might be sent to different child servers.
DistributeOnBatch
mode is most appropriate when you index documents regularly, in batches of approximately equivalent size. This condition ensures that documents are evenly distributed. If you index very large batches at long intervals, documents might be unevenly distributed initially, until enough batches have been indexed so that each child server has processed a similar number of documents.
In DistributeOnBatch
mode, index actions other than DREADD are sent to all child servers.
Type: | Boolean |
Default: | False |
Required: | No |
Configuration Section: | Server |
Example: | DistributeOnBatch=True
|
See Also: | MirrorMode |