KillDuplicates

This parameter determines how handles duplicate documents. It allows you to prevent the same document or document content from being stored in more than once.

Type one of the following options. You can post-fix any of these options with =2, to apply the KillDuplicates process to all databases (rather than only the database into which the current IDX or XML file is being indexed):

NONE Duplicate documents are allowed in and are not replaced or deleted.
REFERENCE If the document being indexed has the same DREREFERENCE field value as a document that already exists in the , deletes the existing document and replaces it with the new document.
REFERENCEMATCHN If the content of the document being indexed is more than N percent similar to the content of a document that already exists in the database, deletes the existing document and replaces it with the new document.
FieldName

If the document being indexed contains a FieldName Reference field with the same value as the FieldName Reference field in a document that already exists in , deletes the existing document and replaces it with the new document.

To specify multiple Reference fields, separate the fields with a plus sign (+) or a space. deletes documents containing any of the specified fields with identical content.

Fields are identified as Reference fields by field processes in the configuration file. If you use a FieldName Reference field to eliminate duplicate documents, automatically reads any fields listed alongside this field for the PropertyFieldCSVs parameter in the field process, and also uses these fields to eliminate duplicate documents. If you want to define multiple reference fields but do not want them all to be used for document elimination, you must set up multiple field processes (refer to the IDOL Server Administration Guide).

If you do not set KillDuplicates, it defaults to the option specified for KillDuplicates in the IDOL server configuration file [Server] section.

Note: When you are using the with DistributeSendMinimal mode, sends a minimal representation to all child servers to allow deduplication. To deduplicate on FieldName you must configure a field process in the configuration file with the fields that you want to use to deduplicate. then includes these fields in the representation it sends to its child servers. By default, it sends only the DREREFERENCE.

For more information about configuring a field process, refer to the IDOL Server Administration Guide.

Actions:

DREADD

DREADDDATA

Type: String
Default:  
Example: KillDuplicates=REFERENCE
See Also:

KeepExisting

KillDuplicatesDB


© 2013 Hewlett-Packard Development Company, L.P.