Open topic with navigation
The previous sections have mainly considered the core IDOL Server text processing components, and distribution. IDOL also includes a large number of other components that allow you to analyze and process unstructured data, which you can use in combination with the text processing, or independently. This section describes the main components, and how you might want to include them in your IDOL system.
License Server is an essential part of any IDOL installation. It manages the licenses for your IDOL products and components.
When you purchase IDOL, HPE Big Data support provide a license key. The license key is bound to a particular MAC address and port, which determines where you must install your License Server. The machine where you install License Server must be accessible over the network to the machines that contain the IDOL services that you want to license.
The only limit to the number of services that a single License Server can manage is your license capacity. In many cases you need only one License Server for your whole setup. However, in some cases you might want to use multiple license servers.
You might want to have multiple License Servers to license separate environments. For example, if you have a production and a test environment, these might have different License Servers. Alternatively, you might want to have a system where each machine that hosts IDOL services has its own License Server.
In these cases, each License Server must have its own license key, with a fixed MAC address and port.
If you want to change or update your system, for example to include new components or features that change your license key, you must update every affected License Server. Similarly, if you need to change the host or port of a License Server, you must request a new license key from HPE Big Data support.
License Server is available in the IDOL Server installer. For more information, see Install IDOL, and the License Server Administration Guide.
IDOL Connectors and the Connector Framework Server (CFS) allow you to collect data from different repositories to import into IDOL.
The number of connectors and CFS components that you need depends on the number of repositories you want to collect data from, and the amount of processing that you want to do.
CFS can process data from multiple connectors. For example, if you want to index content from a few small repositories, and the repositories do not change very often, you might set all the connectors up to send content to the same CFS, which indexes into IDOL Server.
In other cases, you might want to set up multiple CFS instances to process data from different connectors. If your repositories change regularly (generating a large number of new files and updates), or if they are very large, you can use one CFS for each connector.
The CFS is available in the IDOL Server installer. For Linux and Microsoft Windows platforms, the IDOL Server installer also includes File System Connector and Web Connector. For Solaris platforms, the installer includes File System Connector and HTTP Connector. Other connectors are available as separate installers.
For more information, see Install IDOL, and refer to the Connector Framework Server Administration Guide, as well as the administration guides for the individual connectors.
The IDOL multimedia components allow you to process and analyze images, video, and audio files.
Media Server analyzes video files and streams, to extract information about their content. Media Server can run analysis operations such as face recognition, speech-to-text, and number plate recognition.
IDOL Speech Server processes audio files, performing tasks such as speech-to-text transcription, speaker identification, and audio search.
You can use these components in combination with the IDOL text processing components. For example, you might want to use Speech Server with CFS to transcribe audio files that you extract from a repository, and then index the text output into the Content component. You can also use multimedia components independently of other IDOL components. For example, you might use Media Server for face recognition from video files.
Each of these components has its own configuration and setup.
When you use a multimedia component with CFS, you must configure CFS with details of the component to use for processing. If necessary, you can use these components with a DAH for load-balancing and failover.
These components are all available in the IDOL Server installer. For more information, see Install IDOL. For more information about the components, refer to the Media Server Administration Guide, and the IDOL Speech Server Administration Guide.
To add secured search for IDOL Server, you might need to install Omni Group Server (OGS). OGS manages security for documents that you index in the Content component, and ensures that users can access only documents that they have permission to see. OGS can collect security information from several different security repositories and store this information, updating it at regular intervals.
When a user starts a session with the IDOL Community component, the client application requests security information. Community retrieves the user security details from OGS. The client application adds the security information to every subsequent query that it sends to the IDOL Content component. Content can then compare the user security details to the document access control lists (ACLs) to determine what type of access to grant.
Omni Group Server is available as a separate installer. For more information, see Security in IDOL, and refer to the IDOL Document Security Administration Guide.
Setup for secured search requires configuration in the Content and Community components, and your connectors. For more information, refer to IDOL Expert.
Query Manipulation Server (QMS) is an additional component that you can use with a Content index to modify user queries and results. For example, you can use QMS to remove certain terms from the query text, set up promotions to return when a user searches for a particular item, or to return a promoted item to a particular position in the search results.
QMS communicates with a Promotion Agentstore, which stores the rules that QMS uses to determine how to modify queries.
In an IDOL architecture, you send the Content actions that you want to modify to QMS, rather than directly to Content. QMS performs any additional processing, and then forwards the actions to Content.
When you have configured distribution with your Content components, you must decide how to include QMS in the distributed architecture.
You can use one QMS component for each Content, and direct the actions to a DAH that distributes to the QMS components.
In a mirror mode configuration with multiple identical Content components, this architecture ensures that QMS is part of the load-balancing or failover that you use for your system.
You can use one QMS component, and then send the Content actions to the DAH after the QMS processing.
In a non-mirror mode configuration where your index is split over multiple Content components, this architecture reduces the number of additional components you need to configure. In a mirror mode configuration, you can use this architecture if you do not need to include load-balancing or failover for the QMS part of the system.
You can use QMS alone or as part of IDOL Data Admin. IDOL Data Admin provides a front-end application for manipulating and monitoring your queries. For more information, see IDOL Data Admin.
QMS is available as part of the IDOL Server installer. For more information, see Install IDOL, and refer to the Query Manipulation Server Administration Guide.
Statistics Server monitors IDOL logs and collects information on different events.
You can use Statistics Server on its own, but typically you install it as part of an IDOL Data Admin installation. IDOL Data Admin uses Statistics Server to collect information about the queries and terms that you use in your IDOL installation. For more information about IDOL Data Admin, see IDOL Data Admin.
Statistics Server is available as part of the IDOL Server installer. For more information, see Install IDOL, and refer to the Query Manipulation Server Administration Guide.
The Controller and Coordinator components form part of a wider monitoring architecture. Usually, you install these components as part of an IDOL Site Admin installation to monitor and control IDOL services. See IDOL Site Admin.
Controller and Coordinator are available as part of the IDOL Server installer. For more information, see Install IDOL.
The Controller component is selected by default in the installer, so that it is automatically deployed when you install components on a new host matchine.
For information on how to install IDOL Site Admin, refer to the IDOL Site Admin Installation Guide. For information about how to use Controller and Coordinator as part of IDOL Site Admin, refer to the IDOL Site Admin User Guide.
Knowledge Graph can index content to create a graph of the relationships between concepts and entities in your data. You can use Knowledge Graph to provide additional tools for manipulating your unstructured data, and to model connections in a different way to the normal IDOL text processing methods.
You can configure CFS to index data into Knowledge Graph, so that you can create a graph from a set of data. Other than this, Knowledge Graph does not interact directly with other IDOL components in your architecture.