DREADDDATA/KillDuplicates options
Type

String

Default

 

Description

You can enter one of the following options to determine how IDOL server handles duplicate text. Note that if you postfix any of these options with =2, the KillDuplicates process is applied to all IDOL server databases (rather than just the database into which the currrent IDX or XML file is being indexed):

NONE
Duplicate documents are allowed in IDOL server and are not replaced or deleted.

NOOP
IDOL server uses the KillDuplicates setting in its configuration fileís [Server] section to determine how it treats duplicate text.

REFERENCE
If the document being indexed has the same DREREFERENCE field value as a document that already exists in IDOL server, IDOL server deletes the existing document and replaces it with the new document.

REFERENCEMATCHN
If the content of the document being indexed is more than N percent similar to the content of a document that already exists in the IDOL server database, IDOL server deletes the existing document and replaces it with the new document.

FieldName
If the document being indexed contains a FieldName Reference field with the same value as the FieldName Reference field in a document that already exists in IDOL server, IDOL server deletes the existing document and replaces it with the new document.

To specify multiple Reference fields, separate the fields with a plus sign (+) or a space. IDOL server deletes documents containing any of the specified fields with identical content.

Note: Fields are identified as Reference fields through field processes in the IDOL server configuration file (see Reference fields in the IDOL server manual). If you use a FieldName Reference field to eliminate duplicate documents, IDOL server automatically reads any fields that are listed alongside this field for the PropertyFieldCSVs parameter in the field process, and also uses these fields to eliminate duplicate documents. If you want to define multiple reference fields but donít want them all to be used for document elimination, you need to set up multiple field processes (see Using Reference fields to eliminate duplicate copies of documents during indexing in the IDOL server manual).

If you do not set KillDuplicates, it defaults to the option specified for KillDuplicates in the IDOL server configuration fileís [Server] section.

Example

To set the required option for the KillDuplicates parameter, append it directly to the #DREENDDATA tag:

DREADDDATA?[optionalParameters]Data#DREENDDATAREFERENCE\n\n

In this example, KillDuplicates is set to REFERENCE.

See also

KillDuplicatesDB

KeepExisting