Exclusion

Set Exclusion to True to specify that you want to discard any previous pre-filter windows that match the associated pre-filter Regex or ResourceFile parameter.

Eduction runs your pre-filter tasks in the configured order. You can use an Exclusion to exclude match windows that were found by a previous pre-filter. Eduction compares any candidate windows that have been found by an earlier task to the regex or dictionary for the exclusion task. It discards any windows that match the exclusion filter.

For example, if you create a pre-filter that finds numbers to find potential address matches, you can add a subsequent exclusion to remove numbers that are part of dates:

[Eduction]
PrefilterTask0=NumberWordPrefilter
PrefilterTask1=ExcludeDatesPrefilter
PrefilterTask2=StreetMarkersPrefilter

[NumberWordPrefilter]
Regex=\p{N}+,?\p{Z}+\p{Lu}\p{L}
WindowCharsBeforeMatch=100
WindowCharsAfterMatch=100

[ExcludeDatesPrefilter]
Regex=\b([1-9]|[12]\d|3[01]) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)
Exclusion=True

[StreetMarkersPrefilter]
ResourceFile=prefilter/address_street_markers.dpf
WindowCharsBeforeMatch=100
WindowCharsAfterMatch=100

The NumberWordPrefilter task might find potential matches for dates such as 15 March. The subsequent ExcludeDatesPrefilter removes windows that include these dates, so that they are not checked against the Eduction entities.

The third task attempts to find windows based on a dictionary of street markers. Because this task is configured after the exclusion filter, it finds input text such as 15 March Street that had been removed by the exclusion.

NOTE: The parameters that control the window size (WindowCharsAfterMatch and WindowCharsBeforeMatch) are not relevant to pre-filters that have Exclusion set to True.

Type: Boolean
Default: False
Required: No
Configuration Section: MyPreFilterTask
Example: Exclusion=True
See Also:

Regex

ResourceFile