KillDuplicatesOption
For the DREADDDATA index action, you can add the KillDuplicates option directly to the end of the #DREENDDATA
string at the end of your data. For example, #DREENDDATAREFERENCE
uses the REFERENCE
option to remove duplicates.
NOTE: If you set the KillDuplicates action parameter as well, IDOL Server ignores the #DREENDDATA
option.
This parameter determines how IDOL Server handles duplicate documents. It allows you to prevent the same document or document content from being stored in IDOL Server more than once.
Use one of the following options:
NONE
|
Duplicate documents are allowed in IDOL Server and are not replaced or deleted. |
REFERENCE
|
If the document being indexed has the same DREREFERENCE field value as a document that already exists in the IDOL Server, IDOL Server deletes the existing document and replaces it with the new document. |
REFERENCEMATCHN
|
If the content of the document being indexed is more than NOTE: This method can deduplicate only on documents that are already synced in the IDOL Server index. It cannot deduplicate similar documents in the same index job. |
FieldName
|
If the document being indexed contains a To specify multiple Reference fields, separate the fields with a plus sign (+) or a space. IDOL Server deletes documents that contain any of the specified fields with identical content. You must percent-encode any punctuation characters in the field name. NOTE: Fields are identified as Reference fields by field processes in the IDOL Server configuration file. If you use a |
ReferenceField,GREATER:VersionField
|
If the document being indexed contains a
NOTE: When you index IDX documents, for the version comparison to work correctly, the value in the field that you use as the #DREFIELD MyField="N" IDOL Server treats existing documents with a missing or non-numeric value in the |
You can postfix any of these options with =2
, to apply the KillDuplicates
process to all IDOL Server databases (rather than only to the database into which the current IDX or XML file is being indexed).
If you do not set KillDuplicatesOption
, it defaults to the option specified for KillDuplicates in the IDOL Server configuration file [Server]
section. You can also set the following option:
NOOP
|
IDOL Server uses the KillDuplicates setting in its configuration file [Server] section to determine how it treats duplicate text. |
Actions: | DREADDDATA |
Type: | String |
Default: | |
Example: | To set the required KillDuplicates option, append it directly to the #DREENDDATA tag:DREADDDATA?[optionalParameters]Data#DREENDDATAREFERENCE\n\n In this example, KillDuplicates is set to REFERENCE . |
See Also: | KeepExisting
KillDuplicates KillDuplicatesDB |