DistributeByReference
 
Type

Boolean

Default

false

Required

no

Configuration section

[Server]

Description

Type true if you want DIH to index data based on the reference of the documents being indexed or replaced in. This option applies to DREADD, DREADDDATA and DREREPLACE index action. To use distribute by reference mode, you must configure standard field processing options to specify the reference field to use to distribute documents. For details, see the IDOL server online help and IDOL Server Administration Guide.

Reference-based indexing achieves efficient data distribution. DIH uses the references to spread the load evenly between all configured servers. In plain non-mirror mode, DIH sends all index actions to all engines and the child engines calculate which documents to index. With reference-based indexing, this logic is done by the DIH, reducing network traffic and load on the children.

If you enable this option, you cannot alter the number of child engines that DIH indexes to.

Note: reference-based indexing may prevent deduplication of documents with different references, so use reference-based indexing only with a KillDuplicates=REFERENCE or KillDuplicates=NONE setting in the [Server] section of the IDOL configuration file.

Note: If you use a setup with chained DIHs in distribute by reference mode, then you must ensure that the number of children at each level is coprime (that is, they have no common factors) or documents are not distributed evenly by the children.

For example, if you have a parent DIH that has two DIH children, each of which has three IDOL server children, documents are distributed properly if all the DIHs are in distribute by reference mode. However, if the parent DIH is distributing by reference to two DIH children, each of which distributes by reference to four IDOL server children, documents are not evenly distributed.

Example

DistributeByReference=true