Open topic with navigation
A distributed setup involves using a Distributed Index Handler (DIH) and Distributed Action Handler (DAH) to route actions to multiple instances of an IDOL component.
This kind of setup is effective for load-balancing among components, as well as for expanding indexes that no longer fit on one machine. DIH and DAH balance indexing and action requests among the appropriate components. You can either set up the distributed system in mirror mode or non-mirror mode.
The DIH distributes index actions (for example, to add or remove documents from your IDOL index). DIH can only distribute to components that have an index port.
The DAH distributes ACI actions. In mirror mode, it can distribute any action to any component. In non-mirror mode, it can distribute most IDOL Content component actions and combine the results of queries from multiple components.
The following sections provide more information about mirror mode and non-mirror mode for the DIH and DAH when distributing to multiple instances of the IDOL Content component.
For more information, refer to the Distributed Index Handler Administration Guide, and the Distributed Action Handler Administration Guide.
In a mirrored setup, you store the same set of data in each instance of the IDOL Content component. Each Content is identical to the others, and you must configure them in the same way.
Run the DIH in mirror mode to ensure uninterrupted service if one of theIDOL Content components becomes inoperable. While one Content is unavailable, the DIH continues to index data into its identical copies, which are also still available to return data for queries. DIH queues the actions for the inoperable Content, and sends them when the Content becomes available again, so that the servers do not become inconsistent.
DIH sends all index actions to all connected IDOL Content components.
In mirror mode, you can configure the DAH to distribute ACI actions in one of two ways:
Load Balancing. The DAH assigns each incoming action to just one of the connected IDOL Content components (using a cumulative predictive algorithm that spreads the action load efficiently).
Failover. The DAH forwards incoming actions to the first Content that you list in the DAH configuration file. If this IDOL Content component stops responding for any reason, the DAH marks it as down and switches to the next Content.
In a non-mirrored system, you distribute the data equally among the Content components.
Run the DIH in non-mirror mode if the amount of data to index is too large for a single Content. You can also separate the resources for each part of the index by setting the Content components up on different machines. This approach can improve the indexing time.
In non-mirror mode, the DAH sends ACI actions to all connected IDOL Servers. You can configure the DAH to combine the results in different ways when it returns them.
You can set up multiple DIH and DAH instances in a chained configuration. For example, a parent DIH or DAH distributes actions to child DIH or DAH servers, which in turn distributes to child Content components.
In this configuration, the parent DIH and DAH distributes actions to child DIH and DAH servers in the same way as it distributes to child Content components. Each child DIH or DAH accepts all Content actions and forwards them.
Some actions may have a different effect when you send them to a child DIH or DAH server rather than a Content component, because the actions goes to multiple Content components.
Chaining provides an extra level of redundancy both at the DIH or DAH, and the Content level. It also distributes network traffic and system load over a larger number of computers. A chained configuration provides a pool of IDOL Servers that are both fault-tolerant for maximum availability and distributed for the best performance.
For more information about chaining distribution servers, and the architectural considerations, refer to IDOL Expert.
A distributed system with stand-alone components uses any combination of IDOL components with the DIH and DAH. You configure the IDOL components, DIH, and DAH separately with individual component configuration files.
This setup is highly flexible. You can distribute as many or as few of the IDOL components as you need to, and you can scale each component with additional instances as required by your usage. For example, if your system has a high load for categorization, you can increase the number of Category components without adding any other components.
The following diagram shows one example scenario. In this case, there are two Content components, two Community components, and two Category components.
In this example:
You send actions for Content (such as
Query) to Content DAH, which distributes actions between the two Content components.
You send actions for Community (such as
UserRead) to Community DAH, which distributes actions between the two Community components.
You send actions for Category (such as
CategoryQuery) to Category DAH, which distributes actions between the two Category components.
DAH cannot distribute all ACI actions in non-mirror mode, so the Community and Category components in this example must be mirrored.
Agentstore is not shown in this example, but you could use a single Agentstore for all the Category and Community components, or you can have Category and Community connect to an Agentstore DIH and Agentstore DAH. Agentstore indexes are not usually large, so you might not need to split the index into multiple Agentstore components, but you can create mirrored Agentstores for redundancy.
If you require only the basic indexing and retrieval functionality of IDOL Server, a more minimalist setup may be suitable. In this case, you can set up a number of Content servers, with DIH and DAH servers distributing index and ACI actions between them.:
Depending on the size or requirements of the system, you can either use mirror mode for fault tolerance, or non-mirror mode for performance. You can also use chained DIH and DAH servers to allow for both.