The following procedures describe how to configure consistent hashing mode in your DIH and child IDOL servers.
The following procedure describes the configuration changes to the DIH that enable consistent hashing mode. To use consistent hashing, you must configure DIH in DistributeByReference
or DistributeByField
mode.
After you configure the number of VirtualNodes
and Replicas
for your system, you cannot change these values. If you want to change these numbers, you must use a clean DIH installation and reindex all your content.
For more information about these parameters, refer to the DIH Reference.
In the [Server]
section, set the UseConsistentHashing
parameter to True
. In a unified IDOL configuration, set this parameter in the [DistributionSettings]
configuration section. For example:
[Server] UseConsistentHashing=True
[ConsistentHashing]
section.Set the VirtualNodes
parameter to the number of virtual nodes that you want to create. DIH rounds this value up to the nearest power of two (for example, if you set VirtualNodes
to 4000
, it creates 4,096 virtual nodes.
Micro Focus recommends that you set the value of VirtualNodes
to be an order of magnitude higher than the number of child servers that you expect to use.
The minimum value is the higher number of 33 (that is, 64 nodes), or the number of child servers in your system. If you create fewer virtual nodes than there are child servers, DIH does not start. In general, Micro Focus recommends that you do not reduce the default value of virtual nodes (4,096).
Replicas
parameter to the number of identical copies of each document that you want to index. When you configure replicas, DIH copies the documents in a particular virtual node to two or more child servers. For more information, refer to the Distributed Index Handler Reference.The following procedure describes the configuration changes to the child IDOL servers that are required for consistent hashing mode.
In the [FieldProcessing]
section, add a new field process at the bottom of the list, and increase the value of the Number
parameter by one. For example:
[FieldProcessing] Number=4 0=SetIndexFields 1=SetReferenceFields 2=SetSectionBreakFields 3=SetVNodeReferenceField
You must create a new field process, rather than adding the field to an existing reference process.
Create a new configuration section for the field process. Set PropertyFieldCSVs
to */DREVNODE
, and set Property
to the name of a new property. For example:
[SetVNodeReferenceField] Property=VNodeReferenceField PropertyFieldCSVs=*/DREVNODE
Create a configuration section for the corresponding property. Set the ReferenceType
and TrimSpaces
properties to True
. You might also want to use the HiddenType
property for this field. For example:
[VNodeReferenceField] ReferenceType=True TrimSpaces=True HiddenType=True
In consistent hashing mode when you have not configured replicas, you do not need to make any changes to the DAH configuration, except when you add or remove child servers.
When you configure replicas, the DIH distributes identical copies of the virtual nodes between its child servers, ensuring that it does not assign the copies to the same child server. Unlike full server mirroring, the DAH does not know where each of the copies are. In this case, you must:
For more details about these settings, refer to the DAH Administration Guide and IDOL Server Reference.
In addition, when you send a GetQueryTagValues
action to the DAH in a system that uses replicas, the document counts for tags include a value for each copy of the document that exists (that is, the action counts each replica as a separate document).
|