KillDuplicatesOption

Note: For the DREADDDATA index action, you can add the KillDuplicates option directly to the end of the #DREENDDATA string at the end of your data. For example, #DREENDDATAREFERENCE uses the REFERENCE option to remove duplicates.

If you set the KillDuplicates action parameter as well, ignores the #DREENDDATA option.

This parameter determines how handles duplicate documents. It allows you to prevent the same document or document content from being stored in more than once.

Type one of the following options. You can post-fix any of these options with =2, to apply the KillDuplicates process to all databases (rather than only the database into which the current IDX or XML file is being indexed):

NONE Duplicate documents are allowed in and are not replaced or deleted.
REFERENCE If the document being indexed has the same DREREFERENCE field value as a document that already exists in the , deletes the existing document and replaces it with the new document.
REFERENCEMATCHN If the content of the document being indexed is more than N percent similar to the content of a document that already exists in the database, deletes the existing document and replaces it with the new document.
FieldName

If the document being indexed contains a FieldName Reference field with the same value as the FieldName Reference field in a document that already exists in , deletes the existing document and replaces it with the new document.

To specify multiple Reference fields, separate the fields with a plus sign (+) or a space. deletes documents containing any of the specified fields with identical content.

Fields are identified as Reference fields by field processes in the configuration file. If you use a FieldName Reference field to eliminate duplicate documents, automatically reads any fields listed alongside this field for the PropertyFieldCSVs parameter in the field process, and also uses these fields to eliminate duplicate documents. If you want to define multiple reference fields but do not want them all to be used for document elimination, you must set up multiple field processes (refer to the IDOL Server Administration Guide).

If you do not set KillDuplicatesOption, it defaults to the option specified for KillDuplicates in the IDOL Server configuration file [Server] section. You can also set the following option:

NOOP IDOL Server uses the KillDuplicates setting in its configuration file [Server] section to determine how it treats duplicate text.
Actions: DREADDDATA
Type: String
Default:  
Example:

To set the required KillDuplicates option, append it directly to the #DREENDDATA tag:

DREADDDATA?[optionalParameters]Data#DREENDDATAREFERENCE\n\n

In this example, KillDuplicates is set to REFERENCE.

See Also:

KeepExisting

KillDuplicates

KillDuplicatesDB


© 2013 Hewlett-Packard Development Company, L.P.