Other IDOL Components
The previous sections have mainly considered the core IDOL Server text processing components, and distribution. IDOL also includes a large number of other components that allow you to analyze and process unstructured data, which you can use in combination with the text processing, or independently. This section describes the main components, and how you might want to include them in your IDOL system.
License Server
License Server is an essential part of any IDOL installation. It manages the licenses for your IDOL products and components.
When you purchase IDOL, Micro Focus Big Data support provide a license key. The license key is bound to a particular MAC address and port, which determines where you must install your License Server. The machine where you install License Server must be accessible over the network to the machines that contain the IDOL services that you want to license.
The only limit to the number of services that a single License Server can manage is your license capacity. In many cases you need only one License Server for your whole setup. However, in some cases you might want to use multiple license servers.
You might want to have multiple License Servers to license separate environments. For example, if you have a production and a test environment, these might have different License Servers. Alternatively, you might want to have a system where each machine that hosts IDOL services has its own License Server.
In these cases, each License Server must have its own license key, with a fixed MAC address and port.
NOTE: If you want to change or update your system, for example to include new components or features that change your license key, you must update every affected License Server. Similarly, if you need to change the host or port of a License Server, you must request a new license key from Micro Focus Big Data support.
License Server is available in the IDOL Server installer. For more information, see Install IDOL, and the License Server Administration Guide.
Connectors and Connector Framework Server
IDOL Connectors and the Connector Framework Server (CFS) allow you to collect data from different repositories to import into IDOL.
- Connectors access a particular repository, and extract the content to send to CFS. Each connector is specific to a particular repository, rather than a particular file type. For example, you must use File System Connector to access a file system, and Web Connector to access files from the Internet, but both connectors can extract any available file content.
- Connector Framework Server processes the files from your connectors. CFS uses IDOL KeyView and the IDOL multimedia components to extract text from files, which it then sends to your IDOL index (the Content or DIH component). CFS can also perform data enrichment before you index to make it easier to retrieve information.
The number of connectors and CFS components that you need depends on the number of repositories you want to collect data from, and the amount of processing that you want to do.
CFS can process data from multiple connectors. For example, if you want to index content from a few small repositories, and the repositories do not change very often, you might set all the connectors up to send content to the same CFS, which indexes into IDOL Server.
In other cases, you might want to set up multiple CFS instances to process data from different connectors. If your repositories change regularly (generating a large number of new files and updates), or if they are very large, you can use one CFS for each connector.
The CFS is available in the IDOL Server installer. The IDOL Server installer also includes File System Connector and Web Connector. Other connectors are available as separate installers.
For more information, see Install IDOL, and refer to the Connector Framework Server Administration Guide, as well as the administration guides for the individual connectors.
IDOL NiFi Ingest
IDOL NiFi Ingest provides an alternative to Connector Framework Server, with a graphical user interface to help you easily set up and configure your ingestion stream.
To use IDOL NiFi Ingest, you must install Apache NiFi and the IDOL NiFi Ingest package. You can then configure your ingest stream in the NiFi interface, and update your connector configurations to ingest into NiFi, rather than CFS.
For more information, refer to the NiFi Ingest Help.
Multimedia
IDOL Media Server allows you to process and analyze images, video, and audio files.
Media Server analyzes video files and streams, images, and audio to extract information about their content. Media Server can run analysis operations such as face recognition, number plate recognition, speech-to-text, and speaker identification.
You can use this component in combination with the IDOL text processing components. For example, you might want to use Media Server with CFS to transcribe audio files that you extract from a repository, and then index the text output into the Content component. You can also use Media Server independently of other IDOL components, for example for face recognition from video files.
When you use Media Server with CFS, you must configure CFS with details of the component to use for processing. If necessary, you can use these components with a DAH for load-balancing and failover.
Media Server is available in the IDOL Server installer. For more information, see Install IDOL. For more information about the components, refer to the Media Server Administration Guide, and the Media Server Reference.
Secured Search
To add secured search for IDOL Server, you might need to install Omni Group Server (OGS). OGS manages security for documents that you index in the Content component, and ensures that users can access only documents that they have permission to see. OGS can collect security information from several different security repositories and store this information, updating it at regular intervals.
When a user starts a session with the IDOL Community component, the client application requests security information. Community retrieves the user security details from OGS. The client application adds the security information to every subsequent query that it sends to the IDOL Content component. Content can then compare the user security details to the document access control lists (ACLs) to determine what type of access to grant.
Omni Group Server is available as a separate installer. For more information, see Security in IDOL, and refer to the IDOL Document Security Administration Guide.
Setup for secured search requires configuration in the Content and Community components, and your connectors. For more information, refer to IDOL Expert.
Query Manipulation Server
Query Manipulation Server (QMS) is an additional component that you can use with a Content index to modify user queries and results. For example, you can use QMS to remove certain terms from the query text, set up promotions to return when a user searches for a particular item, or to return a promoted item to a particular position in the search results.
QMS communicates with a Promotion Agentstore, which stores the rules that QMS uses to determine how to modify queries.
In an IDOL architecture, you send the Content actions that you want to modify to QMS, rather than directly to Content. QMS performs any additional processing, and then forwards the actions to Content.
When you have configured distribution with your Content components, you must decide how to include QMS in the distributed architecture.
-
You can use one QMS component for each Content, and direct the actions to a DAH that distributes to the QMS components.
In a mirror mode configuration with multiple identical Content components, this architecture ensures that QMS is part of the load-balancing or failover that you use for your system.
-
You can use one QMS component, and then send the Content actions to the DAH after the QMS processing.
In a non-mirror mode configuration where your index is split over multiple Content components, this architecture reduces the number of additional components you need to configure. In a mirror mode configuration, you can use this architecture if you do not need to include load-balancing or failover for the QMS part of the system.
You can use QMS alone or as part of IDOL Data Admin. IDOL Data Admin provides a front-end application for manipulating and monitoring your queries. For more information, see IDOL Data Admin.
QMS is available as part of the IDOL Server installer. For more information, see Install IDOL, and refer to the Query Manipulation Server Administration Guide.
Statistics Server
Statistics Server monitors IDOL logs and collects information on different events.
You can use Statistics Server on its own, but typically you install it as part of an IDOL Data Admin installation. IDOL Data Admin uses Statistics Server to collect information about the queries and terms that you use in your IDOL installation. For more information about IDOL Data Admin, see IDOL Data Admin.
Statistics Server is available as part of the IDOL Server installer. For more information, see Install IDOL, and refer to the Query Manipulation Server Administration Guide.
Service Control
The Controller and Coordinator components form part of a wider monitoring architecture. Usually, you install these components as part of an IDOL Site Admin installation to monitor and control IDOL services. See IDOL Site Admin.
- Controller monitors the IDOL services on a single host. In the typical case where you have your IDOL system distributed across multiple machines, you include a Controller server on each machine. Each Controller reports information to the central Coordinator server.
- Coordinator collects the monitoring information from your Controller servers. You have one Coordinator for the IDOL system that you want to monitor. You can then use the IDOL Site Admin application.
Controller and Coordinator are available as part of the IDOL Server installer. For more information, see Install IDOL.
NOTE: The Controller component is selected by default in the installer, so that it is automatically deployed when you install components on a new host matchine.
For information on how to install IDOL Site Admin, refer to the IDOL Site Admin Installation Guide. For information about how to use Controller and Coordinator as part of IDOL Site Admin, refer to the IDOL Site Admin User Guide.
Knowledge Graph
Knowledge Graph can index content to create a graph of the relationships between concepts and entities in your data. You can use Knowledge Graph to provide additional tools for manipulating your unstructured data, and to model connections in a different way to the normal IDOL text processing methods.
You can configure CFS to index data into Knowledge Graph, so that you can create a graph from a set of data. Other than this, Knowledge Graph does not interact directly with other IDOL components in your architecture.