High Availability in IDOL

This page describes some of the methods you can use to set up a high availability deployment of IDOL Server, CFS, and Connectors.

IDOL Content Component

You can create mirrored instances of the IDOL Content component to provide redundancy if one instance becomes unavailable. Mirrored Content components receive identical data and updates, so that queries can fail over between them. The Distributed Index Handler (DIH) and Distributed Action Handler (DAH) components distribute index actions and query actions to these mirrored servers.

For a setup currently using a single Content component, you can replace the Content component with two or more mirrored servers, and a DIH and DAH configured in mirror mode.

For a setup that already uses DIH and DAH, you can add mirroring by using server groups, without adding more DIH and DAH components to the architecture. In non-mirror modes, you can configure multiple hosts and ports in a single [IDOLServerN] configuration section. For example:

[IDOLServer0]
Host=primaryhost0.domain.org,failoverhost0.domain.org
Port=9010,9010

The DIH and DAH treat the servers at primaryhost0:9010 and failoverhost0:9010 as mirrors.

DIH duplicates index actions to all mirrored servers in a group. DAH sends query actions to one server in each group. You can configure DAH to distribute actions in a mirror group in failover mode, or load balancing mode, by using the DistributionMethod configuration parameter.

In the failover distribution method, if a child server becomes unavailable, DAH switches to the next equivalent child server. When the original child server comes online again, DAH marks it as available. However, it does not switch back to this child server unless the current child server becomes unavailable, or unless you manually change the primary child server by using the EngineManagement action. For more information, refer to the Distributed Action Handler Reference.

Use Mirrored Servers

When you replace a single server with mirrored servers, it does not require many changes to your application logic. However, you must consider the following points:

  • Documents do not necessarily have the same document ID in each server, so you must always refer to the documents by reference.
  • Document indexing and index synchronization might not precisely coincide in the mirrored servers, so at any given time the number of documents available for search might differ. If you use DAH in load balancing mode, you might see responses and numbers of results vary as it sends requests to alternate Content servers. For this reason, and other reasons such as data locality, you might prefer to configure the DAH to use the failover distribution method.
  • Stored state information can be shared between mirrored child servers. If you want to share stored states, configure the StateReferencePath configuration parameter for all mirrored servers to use a shared storage location that all the servers can access. You must also ensure that all stored states use references rather than document IDs, by using the StoredStateField action parameter.

Availability of DIH and DAH

When you use the DIH and DAH to distribute index actions and queries, you might need to take additional steps to ensure that the DIH and DAH components are also highly available.

DAH

When you use a DAH to query Content components, the DAH does not preserve much internal state. For this case, you can set up multiple redundant DAHs with identical configuration, and route queries to the different DAH components using a hardware load balancer.

An exception to this rule is for stored states. DAH maintains an internal mapping between the tokens it receives from its child servers and the tokens that it returns to users. If you fail over to a second DAH, the new DAH does not have access to the same mappings database, and cannot return information about existing stored states. For this reason, if your application uses stored states, you might want to route queries to a single DAH as long as it is available. If it becomes unresponsive, and the application switches to another DAH, the application must rebuild any stored states that it is currently using.

DIH

Unlike the DAH, the DIH stores a lot of internal state information about incoming and outgoing queues. If your applications do not rely on having very recent content, it might not be a problem for indexing to become temporarily unavailable while the DIH is offline. Sending components such as CFS can queue index jobs until the DIH is ready to receive them.

If your application depends on current content, you can set up an identically-configured DIH for failover. However, in this case, there might be some index actions that are queued for processing by the failed DIH and not yet sent to its child servers. The child servers do not receive these index actions and associated content until the primary DIH is brought back online.

Other IDOL components

Category and Community

You can mirror other IDOL components, such as the Category and Community components, behind a DAH configured in mirror mode. For Category and Community, ideally each instance must have a local Agentstore Content component. If you have a Category and Community component on the same host, these components can share the local Agentstore.

Category and Community must also have access to the data index. For a high availability setup, you can configure the [DataDRE] configuration for these components to use the top-level DAH to distribute the requests to your data index Content components.

TIP: Where you have multiple redundant top-level DAHs, OpenText recommends that you set up a local DAH in mirror mode, configured with a failover distribution method to distribute to the redundant DAHs.

DAH can send most actions to a single Category or Community child server. However, some actions change the state of the component, and so the DAH must send these actions to all instances of the component to maintain the mirrored state. The DAH stores details of these state-changing actions (SCAs) in the sca.dat file. If one of the component mirrors becomes unavailable, DAH queues a limited number of SCAs to replay when this component comes back online.

When you use a DAH to distribute to mirrored Category and Community components, OpenText recommends that you configure the DAH configuration file [StateChangingActions] section with the following options:

[StateChangingActions]
MapCategoryIDs=False
MapCommunityIDs=False

For more information, refer to the Distributed Action Handler Reference.

View

You can replace a single View server component with two or more servers behind a DAH in mirror mode, configured with the failover distribution method.

For load-balancing, you can set up multiple View servers with the DAH in mirror mode with the reference distribution method. In this case, you can configure failover as well by replacing each single View server with a dedicated mirror mode DAH, configured for a failover distribution method, and two or more child View servers.

The reference distribution DAH does not store state information, so you can also duplicate the DAH hierarchy for redundancy, pointing at the same set of View servers.

Asynchronous ACI Servers

In mirror mode, you can distribute asynchronous actions to asynchronous ACI servers. For DAH to distribute asynchronous actions, you must configure it to route requests according to the async token, and to merge the responses of QueueInfo actions. The following example shows the basic required configuration:

[Server]
MirrorMode=true
AsynchronousCommands=ListOfActions

[QueryTemplate]
QueueInfo=queueinfo.xsl

ListOfActions is a comma-separated list of the asynchronous actions that you want to distribute. The QueueInfo query template, queueinfo.xsl is an XSLT template that controls how DAH merges QueueInfo action responses. This template is included in your installation when you use the full IDOL Server installer to install the distributed handlers. You must also configure DAH to use XSLT templates (see Use XSLT in IDOL).

NOTE: DAH can automatically determine which child server handled a request from the asynchronous action token only. It does not maintain an internal mapping of tokens to server instances. This behavior means that you can set up failover between multiple DAH components to distribute to the same set of asynchronous servers.

Connectors and CFS

You can use the Distributed Connector to provide redundancy for your connectors. The Distributed Connector distributes connector actions (Collect, Hold, and so on) from an end application to a connector or series of connectors. If one connector stops responding, it switches to using a different connector.

When a connector registers with a Distributed Connector, it must join a connector group. A connector group is a group of similar connectors. The connectors in a group must be of the same type (for example, all HTTP Connectors), and must be able to access the same repository. The Distributed Connector can handle multiple connector groups simultaneously.

Generally, OpenText recommends that only one connector retrieves data from a repository at any one time, because having more can affect the responsiveness of the repository. You can send these fetch tasks to the Distributed Connector, which selects the appropriate connector from the appropriate group.

For more information about setting up connectors with Distributed Connector, refer to the Distributed Connector Administration Guide and the documentation for the connector that you are using.

Each Connector can only send data to one ingestion target, but you can point the connector to a DAH, which can mirror actions to multiple CFS instances. In this case, see the Other IDOL components section for information about using asynchronous ACI servers with a DAH.