Field Standardization

Field standardization modifies documents so that they have a consistent structure and consistent field names. You can use field standardization so that documents indexed into IDOL through different connectors use the same fields to store the same type of information. Field standardization only modifies fields that are specified in a dictionary, which is defined in XML format. A standard dictionary, named dictionary.xml, is supplied with IDOL Connectors, CFS, and NiFi Ingest.

In most cases you should not need to modify the standard dictionary, but you can modify it to suit your requirements or create dictionaries for different purposes. By modifying the dictionary, you can configure rules that modify documents before they are indexed. For example, you can move fields, delete fields, or change the format of field values.

This section describes the XML elements that you can use to build or customize a field standardization dictionary.

Element Description
AddField Adds a field to the document metadata.
Attribute Represents an existing document metadata field attribute.
Copy Copies a field.
Delete Deletes a field.
Field Represents an existing document metadata field.
GetName Stores the name of the current context field or attribute in a variable.
GetValue Stores the value of the current context field or attribute in a variable.
IfAttribute Makes operations conditional on the presence of a metadata field attribute.
IfField Makes operations conditional on the presence of a metadata field.
IfVar Makes operations conditional on whether a variable exists and optionally matches a given value.
Includable Defines operations that can be used elsewhere in the field standardization dictionary.
Include Includes operations defined elsewhere in the field standardization dictionary.
Move Moves or renames a field.
NewField Builds a field but does not immediately change the document metadata.
Product Contains field standardization rules for a specific product.
SetAttribute Adds or updates an attribute on the current context field.
SetValue Updates the value of the current context field or attribute.
SetVar Sets the value of a variable.
ValueFormat Converts a field or attribute value from one format to another.