Regenerating an IDOL data indexes rebuilds the index of stored information for optimized fields (MatchType fields, NumericType fields, and so on) from the textual document field values held in the server. You can do this to reconcile the index or indexes with the document data if:
validation of one or more indexes has failed, indicating that the data is no longer in a consistent state
you have changed the server [FieldProcessing]
or [Properties]
configuration to:
make an existing field an optimized type.
remove an optimized field.
set or clear the NumericIntegerOnly
setting for a numeric type field.
In addition, you might need to regenerate fields if you have indexed ReferenceMemoryMappedType
fields out of order, with some child fields indexed before their parents.
In these cases, the alternative to regenerating is to reindex all documents. Regenerating is generally significantly faster, as only the requested indexes regenerate.
You can regenerate a data index by using one of the following methods:
DREREGENERATE
index action. This option allows you to rebuild the index while the server is running. The following tables describe the regeneration options for different indexes and field types.
Index Type | Validation Repair | Update Configuration | Configuration Parameter | DREREGENERATE Type |
---|---|---|---|---|
BitFieldType
|
![]() |
RegenerateBitFieldIndex
|
BitField
|
|
CountType
|
![]() |
RegenerateCountIndex
|
Count
|
|
GeospatialType
|
![]() |
RegenerateGeospatialIndex
|
Geospatial
|
|
MatchType
|
![]() |
RegenerateMatchIndex
|
Match
|
|
NumericDateType
|
![]() |
RegenerateNumericDateIndex
|
NumericDate
|
|
NumericType
|
![]() |
![]() |
RegenerateNumericIndex
|
Numeric
|
ParametricRangeType
|
![]() |
RegenerateParametricIndex
|
Parametric
|
|
ParametricType
|
![]() |
RegenerateParametricIndex
|
Parametric
|
|
ReferenceType
|
![]() |
RegenerateRefIndex
|
RefIndex
|
|
SecurityType
|
![]() |
RegenerateSecIndex
|
SecIndex
|
|
SortType
|
![]() |
RegenerateSortFieldIndex
|
SortField
|
|
Unstemmed indexWildcard searches (and some advanced search techniques) use the index of unstemmed terms. This index contains all the exact terms that occur in documents. |
![]() |
RegenerateUnstemmedIndex
|
The DREREGENERATE
index action allows you to change the field configuration and regenerate the index in one action. You can also use it to repair the Reference and Security indexes after a validation failure.
For example:
DREREGENERATE?FieldProcessingSection=SetMatchFields&AugmentFieldCSVs=*/NewMatchField,*/SpecialMatch&DiminishFieldCSVs=*/ExistingMatchField
By default, DREREGENERATE
automatically detects the configuration changes that you make and regenerates the indexes accordingly. You can also explicitly select the indexes to regenerate. For example:
http://12.3.4.56:20001/DREREGENERATE?&Type=Numeric,RefIndex
For more information, refer to the IDOL Server Administration Guide and the IDOL Server Reference.
When the index action runs, IDOL runs through all stored documents, and loads the values of any fields to regenerate from disk. It inserts these values into new index structures.
When the process is complete, IDOL Server deletes any existing indexes and replaces them with the newly built versions.
When you regenerate by updating the configuration file, you must restart the server. Regenerating the indexes also results in an increased startup time.
To regenerate a data index
Stop IDOL Server.
Make any required changes to the [FieldProcessing]
or [Properties]
configuration sections.
Set the required Regenerate*Index
settings to True
in the [Server]
section of the configuration file.
Restart IDOL Server.
At start-up, IDOL runs through all stored documents, and loads the values of any fields to regenerate from disk. It inserts these values into new index structures.
When the process is complete, IDOL Server deletes any existing indexes and replaces them with the newly built versions.
After the server has started, remember to remove or turn off any Regenerate*Index
settings in the configuration file if you do not want the server to regenerate the indexes every time it starts.
Regenerating the index by using DREREGENERATE
is typically I/O intensive, because Content must reread all the documents in your index. Where it can, Content generates a second data structure in memory, and swaps over the files when it has completed the process. This option might be more memory intensive than using the configuration option, but it means that you can continue to query the existing index while regeneration is in progress.
Regenerating the index by using the Regenerate*Index
configuration options requires a server restart. In addition, regenerating indexes also results in an increased startup time. The server is not available for further indexing or querying until the regeneration process is complete.
Regenerating the unstemmed index is an intensive process and generally takes significantly longer than regenerating other indexes (though it is still faster than reindexing the data).
Consider the following points when you use the regeneration options:
The regeneration affects all indexes of the requested type. For example, if you add a field as NumericType in the configuration file, setting RegenerateNumericIndex
to True
regenerates all existing numeric indexes, as well as creating one for the new numeric field.
The original document content must be stored in IDOL Server.
If you have set NodeTableStoreContent
to False
, you can only regenerate a given index if all the fields in question are also explicitly configured as StoredType
.
|