Store Content in IDOL Server > Index Data > Prevent Duplicate Documents > Deduplication Options—KillDuplicates and IndexMode

Deduplication Options—KillDuplicates and IndexMode
Specify deduplication options using the following parameters. IDOL uses them to determine whether documents match.
*
The KillDuplicates parameter specified in either the [Server] section of the IDOL server configuration file or in the DREADD or DREADDDATA index action.
*
The IndexMode configuration parameter in the connector configuration file.
The following options are available for the deduplication parameters:
 
Replaces an existing document with the new document if the document to index has the same value in its DREREFERENCE field.
Replaces the existing document with the new document if the content of the document to index is more than N percent similar to the existing document.
Replaces the existing document with the new document if the document to index contains a ReferenceType field named FieldName that has the same content as the FieldName field in the existing document.
You can specify multiple ReferenceType fields in this option (separated by a plus symbol or space), in which case IDOL server deletes documents that contain any of the specified fields with identical content.
Note that you identify fields as ReferenceType fields through field processes in the IDOL server configuration file. If you list multiple fields in the same PropertyFieldCSVs parameter where you list the FieldName for deduplication, IDOL server uses all the fields to eliminate duplicate documents. If you want to define multiple ReferenceType fields but do not want to use all for duplicate elimination, set up multiple field processes.
Use the KillDuplicates parameter in the [Server] section of the IDOL server configuration file to determine how to treat duplicate documents.
Related Topics 
*
*
When you specify a deduplication option, note that:
*
If you postfix any of these options with =2, IDOL server applies the KillDuplicates process to all databases, rather than just the database into which the current IDX or XML file indexes. For example:
KilDuplicates=REFERENCE=2
*
The setting in the KillDuplicates option in either the DREADD or DREADDDATA index action overrides the setting in the KillDuplicates configuration parameter.