This section lists the resolved issues in the IDOL PII Package version 12.7.
Offsets and offset lengths returned for components were different for EJR grammars compared to their equivalent ECR grammar.
EJR grammars could return more components than equivalent ECR grammar.
When using Eduction in CFS or NiFi, Lua post-processing scripts that renamed the matching entity could drop matches, or return the match under an unexpected field name (such as "_"). For example, the post-processing scripts shipped in the IDOL PII Package rename entities from the combined grammars ending "/all" to correspond to the matched language.
Post-processing replaced the normalized text with empty space for pii/name/given_name/nocontext/all or pii/name/surname/nocontext/all entity matches.
When matching nocontext name components other than given name or surname (such as pii/name/pre_title/all), post-processing could log an error "Error during call to lua function 'processmatch' in script '.../pii_postprocessing.lua': Parameter 2 had the wrong type".
In post-processing for TIN and national ID, an issue with the lua script resulted in some false positive matches when scanning for alternative matches after an initial context or nocontext match had failed.
For TIN and national ID, nocontext matches for countries that did not have a checksum algorithm could receive a score boost in post-processing. Now, these matches receive a score boost only if they are confirmed by checksum.
In the pii_postprocessing.lua
scripts, print()
was used instead of error()
or assert()
in some cases, which could result in errors not being correctly returned when using the Eduction SDK.
name_cjkvt
grammar did not accurately reflect common Japanese given names.