IDOL Core Components

This section describes the common IDOL components.

The following components form the core IDOL text indexing and processing functions. Together, they are often collectively known as IDOL Server. These components are all ACI servers. For more information, refer to the IDOL Server Help.

IDOL Text Processing
Service Description
Content

Indexes, collects, manipulates and stores unstructured and semi-structured text data.
AgentStore
Indexes, collects, manipulates and stores agent and category information. Agentstore is a special configuration of the Content component.

Category

Creates and manages categories, and performs text analysis such as clustering and taxonomy generation.
Community
Stores information about users, roles, and permissions.
View
Converts documents to HTML format for viewing in a Web browser.

The following components form part of a wider IDOL infrastructure. They allow you to license, distribute, monitor, and control your IDOL components. These components are also all ACI servers.

Infrastructure
Service Description
License Server

Distributes and manages licenses for IDOL products and components.

Refer to the License Server Help for more information.

IDOL Proxy
Routes actions to the correct components in a unified IDOL Server, and starts, restarts and stops IDOL components.
DAH

Distributed Action Handler. Distributes action requests across IDOL components.

Refer to the DAH Help for more information.

DIH

Distributed Index Handler. Distributes indexing requests across IDOL Content components.

Refer to the DIH Help for more information.

Controller

Monitors the IDOL services on a host machine, and communicates with Coordinator to report status information. This component is primarily used by IDOL Site Admin.

Refer to the IDOL Site Admin User Guide for more information.

Coordinator

Manages status information from Controllers, and acts as a central point for viewing logs and monitoring the status of the IDOL system. This component is primarily used by IDOL Site Admin.

Refer to the IDOL Site Admin User Guide for more information.

The following components allow you to automatically retrieve content and index into IDOL.

IDOL Ingest
Service Description
NiFi Ingest

Allows you to easily configure and manipulate your data ingest process, from your connectors, to KeyView and other import processes (such as media analysis and Eduction), and your IDOL index. It has a user interface based on Apache NiFi, which you use to configure your whole ingest chain. You can use it to retrieve data from many repositories, use data enrichment (with routing to ensure different file types get appropriate processing), and index into IDOL, or update data in your repositories.

Refer to the NiFi Ingest Help for more information.

CFS

Connector Framework Server. Aggregates data from IDOL connectors and indexes the data into IDOL Server. CFS can also manipulate the IDOL documents sent to it by the IDOL connector (for example, to add additional metadata fields generated by data enrichment tasks). CFS is an ACI server.

Refer to the Connector Framework Server Help for more information.

Connectors

Gather data from different sources for indexing into IDOL. In some cases, connectors can also insert back into the original repositories.

There are connectors for many different repositories. For NiFi based system, each connector is a NiFi processor, which you configure from the NiFi interface. For CFS-based systems, each connector is an ACI server, which you configure separately.

Refer to the NiFi Ingest Help or the Connector Administration guides for more details.

The following components perform additional operations and analysis on unstructured data. You can use many of these components with IDOL text processing, or as part of an independent system. These components are all available as ACI servers, and some are also available in other formats.

Additional Services
Service Description
Media Server

Media Server analyzes video files and streams, images, and audio to extract information about their content. It can run analysis operations such as face recognition, number plate recognition, speech-to-text, and speaker identification.

Media Server is an ACI Server. A lot of Media Server functionality is also available in the NiFi media analysis processors, which you can use with NiFi Ingest.

Refer to the Media Server Help for more information.

Eduction

Extracts entities (small pieces of information) from text, such as names, addresses, and telephone numbers. Eduction is available as an ACI server to use with other IDOL services, as a separate SDK, or embedded in NiFi Ingest and CFS.

Refer to the Eduction User and Programming Guide and the Eduction Server Reference for more information.

Answer Server

Performs Search and Retrieval and Conversations. Answer Server can answer questions from a database of facts, from a stored list of questions such as an FAQ, or by returning a relevant passage from a relevant store of unstructured data. It can also manage ongoing conversations with context, for example to run a chatbot.

Refer to the Answer Server Help for more information.

QMS

Query Manipulation Server. Manages promotions, modifies queries to IDOL Server, and manipulates results from IDOL Server. For example, you can use QMS to remove certain terms from the query text, set up promotions to return when a user searches for a particular item, or to return a promoted item to a particular position in the search results.

Refer to the Query Manipulation Server Help for more information.

Statistics Server

Accumulates events from client applications or from a script that reads IDOL log files, then uses that data to report statistics.

Refer to the Query Manipulation Server Help for more information.

OGS

OmniGroupServer. Collects security information from security repositories and uses it to ensure that users can access only documents that they have permission to access.

Refer to the IDOL Document Security Administration Guide and OmniGroupServer Reference for more information.

Knowledge Graph

Uses connections between IDOL documents to create a set of relationships called a graph, and allows you to explore these connections in your data.

Refer to the Knowledge Graph Help for more information.