Measure

This command measures the precision and recall between extraction runs by comparing the expected results of entity extraction with the actual results.

You create expected results once and keep them as a base reference for ongoing tests. You then generate actual results as required each time you modify a grammar. Edktool compares the two results to generate precision and recall information.

To generate expected results, run edktool -extract, and then revise the generated output file so that it contains the correct matches. From then on, you use edktool -extract to create the actual results, and the measure command compares the two files to generate precision and recall information on an ongoing basis.

The following table describes the parameters for this command.

-e <expectedfile> The expected results file.
-a <actualfile>

The actual results file from subsequent extraction runs with modified grammar files.

-o <resultsfile >

The output file, including the results for precision, recall, and differences.

-q (Optional) Run in quiet mode. In this case, edktool removes all descriptive messages from the output and shows only the differences between the expected and actual output.

For more information on how to use measure to check the effectiveness and performance of your grammar files, see Assess and Measure Eduction Grammars.

Example

The following example compares expected.xml with actual.xml and puts the difference in difference.xml, including precision and recall. Quiet mode is enabled, so all descriptive messages are removed from the output.

edktool m -e expected.xml -a actual.xml -o difference.xml -q