Manage the Index Queue
The DIH maintains an internal queue of unfinished index actions that it receives. By default, the index queue size grows if the DIH needs to accept new index actions (that is, it does not discard jobs that have not been completed).
To tune indexing performance, you can control certain characteristics of the queue.
- You can presize the DIH index queue by setting the IndexQueueInitialSize parameter in the
[IndexQueue]
section of the configuration file. When you know that DIH is going to receive a large number of index jobs, you can use this option to reduce the number of times that DIH resizes the index queue. - You can limit the size of the index queue by setting the IndexQueueMaxPendingItems parameter in the
[IndexQueue]
section of the configuration file. This parameter limits the number of incomplete index actions on the queue. When the queue size reaches this limit, DIH rejects any further index actions. You can override the queue limit for an individual index action by using the IgnoreMaxPendingItems index action parameter. -
You can specify the maximum number of finished (historical) index actions by setting the IndexQueueMaxHistory configuration parameter in the
[IndexQueue]
section of the configuration file. In this case, DIH keeps the most recent finished actions, and removes the oldest. -
By default, the DIH periodically polls each job in the index queue to see whether it is complete. When a job is complete,
-
Specify how often the polling occurs by modifying the PollInterval configuration parameter in the [
Server]
section of the DIH configuration file. The default value is 10 seconds.Frequent polling might help to keep the queue size down if you exceed the number of jobs specified in the IndexQueueMaxHistory configuration parameter. However, polling too frequently might affect indexing performance.
-
You can turn off polling completely by setting the Polling configuration parameter to
False
. In this case, DIH marks the job as complete as soon as it sends it to the child servers. DIH keeps the number of finished jobs specified by IndexQueueMaxHistory.Turn off polling for all child servers by setting Polling to
False
in the[Server]
section. Alternatively, turn off polling for individual child servers by setting Polling toFalse
in the[IDOLServerN]
section.
-
- You can specify a directory for the DIH to use to store index queue status files. By default, it stores the files in the
./main
directory, relative to the DIH executable. Set the Main configuration parameter (in the [Paths]
section of the DIH configuration file) to the full or relative path to the directory to use.