When you run the classifier set optimization task, HPE IDOL Speech Server writes information about the optimization performance to a log file. To specify the log file, use the OutputLog
parameter in the [lidoptimizer]
section of the tasks configuration file (speechserver-tasks.cfg
). To retrieve this log file, use the GetResults
action, and set the Token
action parameter to the token for the optimization task.
The log file shows how the language classifiers are performing on each iteration of the optimization. The <lidoptimize_iteration_record>
summarizes the system performance across all languages for a specific iteration. For example:
<lidoptimize_iteration_record> <iteration>11</iteration> <learning_rate>0.923</learning_rate> <avg_score>0.951</avg_score> <min_score>0.847</min_score> <std_dev>0.0107</std_dev> <wt_nincreased>9</wt_nincreased> <wt_increased>ENUK,ENUS,ESES,ESLA,ARMSA,FRFR,DEDE,NLNL,ITIT</wt_increased> <wt_ndecreased>1</wt_ndecreased> <wt_decreased>PLPL</wt_decreased> <wt_avgchange>0.000939</wt_avgchange> <wt_avgchange_abs>0.000974</wt_avgchange_abs> <result_delta>0.000442</result_delta> <best_so_far>No</best_so_far> </lidoptimize_iteration_record>
The record contains the following elements.
iteration
|
The current iteration of the optimization process. |
learning_rate
|
The current learning rate of the algorithm. The learning rate decreases throughout the process. |
avg_score
|
The average correct score across all languages, with the current weighting. |
min_score
|
The score of the worst performing language classifier. |
std_dev
|
The standard deviation across the classifier scores. |
wt_nincreased
|
The number of languages whose weights were increased. |
wt_increased
|
The language pack codes for the languages whose weights were increased. |
wt_ndecreased
|
The number of languages whose weights were decreased. |
wt_decreased
|
The language pack codes for the languages whose weights were decreased. |
wt_avgchange
|
The average weight change in real terms. |
wt_avgchange_abs
|
The average weight change in absolute terms. |
result_delta
|
The delta in the result when compared to the previous iteration. |
best_so_far
|
Whether this iteration was the best iteration. |
The aim of the optimization process is to optimize the values for avg_score
, min_score
, and std_dev
.
The log file also contains <lidoptimize_lang_record>
items. This item shows the performance of a specific language at a specified optimization iteration. For example:
<lidoptimize_lang_record> <iteration>11</iteration> <lang>HBIL</lang> <weight>1.031</weight> <score>0.947</score> <update_avgTrueClarity>1.105</update_avgTrueClarity> <update_minTrueClarity>1.040</update_minTrueClarity> <update_avgFalseClarity>1.110</update_avgFalseClarity> <update_minFalseClarity>1.041</update_minFalseClarity> <closest_imposter>ARMSA (33.3%)</closestImposter> <weight_pos1>0.934</weight_pos1> <weight_pos2>0.994</weight_pos2> <weight_neg1>1.133</weight_neg1> <weight_neg2>1.069</weight_neg2> <rel_change>0.001</rel_change> <rel_change_str>INCREASED</rel_change_str>k <next_weight>1.032</next_weight> <min_conf>1.011</min_conf> <low_conf>1.105</low_conf> <med_conf>1.157</med_conf> <high_conf>1.204</high_conf> <max_conf>1.318</max_conf> <false_conf>1.000</false_conf> </lidoptimize_lang_record>
The record item shows the existing weight and the new weight for the language, as well as some information on intermediate values used to calculate it. The record also gives information on how the language performed on the last iteration, information about confusability and the closest imposter for the language, and details about the spread of confidence (from the best result to the worst).
iteration
|
The current iteration of the optimization process. |
lang
|
The language that this record represents. |
weight
|
The weight for this language classifier at this iteration. |
score
|
The correct rate for this classifier at this iteration (that is, is the proportion of audio segments for this language that were correctly identified). |
next_weight
|
The new weight for this classifier, for the next iteration. |
min_conf
|
The minimum confidence seen for a true hit for this language. |
low_conf
|
A low confidence value seen for a true hit for this language. |
medium_conf
|
A medium confidence value seen for a true hit for this language. |
high_conf
|
A high confidence value seen for a true hit for this language. |
max_conf
|
The maximum confidence value seen for a true hit for this language. |
false_conf
|
The maximum confidence seen for a false hit with this classifier. |
After all optimization iterations are complete, the <lidoptimize_result>
log item reports the best iteration. For example:
<lidoptimize_result> <best_iteration>4</best_iteration> </lidoptimize_result>
HPE IDOL Speech Server writes the language classifier weights for the best iteration to the classifier list file. When you use this file to run language identification tasks, HPE IDOL Speech Server balances the classifier scores based on the new, trained weights.
|