Open topic with navigation
When you instruct the IDOL Content component to eliminate duplicate document copies at index time using a specific
ReferenceType field (by setting the
KillDuplicates parameter in the IDOL Content component configuration file), it automatically uses any field listed for
PropertyFieldCSVs alongside this
ReferenceType field in the IDOL Content component configuration to eliminate duplicate document copies as well.
However, Content cannot use the same field for deduplication as for the
Combine action parameter, because the
Combine operation clashes (carried out at query time) with Content eliminating duplicate fields. This clash means that, if you want to eliminate duplicate document copies and use the
Combine action parameter, you must set up separate
ReferenceType fields for these processes.
Open the IDOL Content component configuration file in a text editor.
[FieldProcessing] section, add two processes that identify
ReferenceType fields (note that you must set up a field process to identify
ReferenceType fields before you start indexing documents into Content). One of them is used to eliminate duplicate copies of documents, and the other one is used for the
[FieldProcessing] 0=MyFirstProcess 1=MySecondProcess 2=SetUpReferenceFields 3=SetUpMoreReferenceFields
Create a section for the processes that you added, and in each section, create a property for the respective process (you define the property later by setting one or more applicable configuration parameters). Identify the fields that you want to associate with each process.
The properties that you create must not have the same name as the processes.
[MyFirstProcess] Property=MyFirstProperty PropertyFieldCSVs=*/MyField,*/MySecondField [MySecondProcess] Property=MySecondProperty PropertyFieldCSVs=*/MyThirdField [SetUpReferenceFields] Property=ReferenceFields PropertyFieldCSVs=*/DREREFERENCE,*/URL [SetUpMoreReferenceFields] Property=MoreReferenceFields PropertyFieldCSVs=*/DRETITLE
Create a section for each of the properties and specify appropriate configuration settings for each. These configuration parameters define the processes that are applied to all the fields (or all documents that contain the fields) that you previously associated with the processes. For example:
[MyFirstProperty] HiddenType=True [MySecondProperty] Index=True [ReferenceFields] ReferenceType=True TrimSpaces=True [MoreReferenceFields] ReferenceType=True TrimSpaces=True
Save and close the configuration file.
Restart the IDOL Content component for your changes to take effect.
After you index documents into Content, you can use, for example, the
*/DREREFERENCE field to eliminate duplicate copies of documents. (Content then automatically also uses the
*/URL field for deduplication because it is listed alongside
PropertyFieldCSVs.) This leaves you free to use the
*/DRETITLE field for the