Type true
if you want Distributed Index Handler to index data based on the reference of the documents being indexed or replaced in. This option applies to DREADD, DREADDDATA and DREREPLACE index actions. To use distribute by reference mode, you must configure standard field processing options to specify the reference field to use to distribute documents. For details, see the IDOL Server Online Help and IDOL Server Administration Guide.
Reference-based indexing improves the efficiency of data distribution, and reduces the network traffic and load on the child servers. In this mode, Distributed Index Handler uses the document reference to spread the load evenly between all configured servers. With reference-based indexing, Distributed Index Handler sends only the documents that the child server must index, whereas in plain non-mirror mode, Distributed Index Handler sends all index actions to all servers and the child servers calculate the documents to index.
If you enable this option, you cannot alter the number of child servers that Distributed Index Handler indexes to.
When DistributeByReference
is set to true
, DIH always distributes documents according to the reference. If a child server goes down, it queues the documents for that child server, and indexes them when the server becomes available. Unlike simple non-mirror mode, it does not redistribute the documents to available servers.
Note: Reference-based indexing can prevent deduplication of documents with different references, so use reference-based indexing only with a KillDuplicates=REFERENCE
or KillDuplicates=NONE
setting in the [Server]
section of the IDOL configuration file.
If you have chained Distributed Index Handlers in distribute by reference mode, documents might not be distributed evenly by the child servers. Documents are evenly distributed only if the number of child servers at each level are coprime (that is, they have no common factors).
For example, if you have a parent Distributed Index Handler that has two Distributed Index Handler children, each of which has three IDOL server children, documents are distributed properly if all the Distributed Index Handlers are in distribute by reference mode. However, if the parent Distributed Index Handler is distributing by reference to two Distributed Index Handler children, each of which distributes by reference to four IDOL server children, documents are not evenly distributed.
Type: | Boolean |
---|---|
Default: | False |
Required: | No |
Configuration Section: | Server |
Example: | DistributeByReference=true
|
See Also: | Field Processing Configuration Parameters |
|