Use the following parameters to specify deduplication options. The IDOL Content component uses these parameters to determine whether documents match.
The KillDuplicates
parameter specified in either the [Server]
section of the IDOL Content component configuration file or in the DREADD
or DREADDDATA
index action.
The KillDuplicates
parameter specified in the [Indexing]
section of the CFS configuration file.
The following options are available for the deduplication parameters.
NONE
|
Allows duplicate documents in Content index. Content does not replace or delete documents. |
REFERENCE
|
Replaces an existing document with the new document if the document to index has the same value in its DREREFERENCE field. |
REFERENCEMATCHN
|
Replaces the existing document with the new document if the content of the document to index is more than NOTE:
This method can deduplicate only documents that are already synced in the IDOL Server index. It cannot deduplicate similar documents in the same index job. |
FieldName
|
Replaces the existing document with the new document if the document to index contains a You can specify multiple NOTE:
You identify fields as |
ReferenceField,GREATER:VersionField
|
Replaces the existing document with the new document if the document to index contains a
NOTE:
When you index IDX documents, for the version comparison to work correctly, the value in the field that you use as the #DREFIELD MyField="N" IDOL Server treats existing documents with a missing or non-numeric value in the |
NOOP (DREADDDATA only) |
Use the NOTE:
This option is available only for the |
When you specify a deduplication option, note that:
If you postfix any of these options with =2
, Content applies the KillDuplicates
process to all databases, rather than just the database into which the current IDX or XML file indexes. For example:
KillDuplicates=REFERENCE=2
The setting in the KillDuplicates
option in either the DREADD
or DREADDDATA
index action overrides the setting in the KillDuplicates
configuration parameter.
|