Custom Word Databases

The language packs that are available for Media Server are trained with billions of words, but any language model has a finite vocabulary. When you run speech-to-text you might find that some words, such as product names, are not included in the vocabulary. A word that is not in the vocabulary is never detected.

The best way to expand the vocabulary for a speech-to-text analysis task is to create a custom language model (see Custom Language Models), but if you have insufficient text to train a custom language model or you want to add some extra words alongside a custom language model, you can use a custom word database.

A custom word database contains a list of words that you want to add to the vocabulary. To add a custom word you only need to supply the word, but OpenText recommends that you also specify a base word and weight. A base word is a word that exists in the standard language model and could appear in the same context as the custom word. For example, if you want to add a company name "AcmeSoft" to the custom word database, you could specify "Microsoft" as the base word. This instructs Media Server that "AcmeSoft" is expected to appear in the same context or similar contexts as "Microsoft". The weight is a multiplier that specifies how likely the word is to appear, relative to the base word. For example, if the word you are adding is only slightly less likely to appear than the base word, you might set a weight of 0.8. If the word is much less likely to appear you might set a weight of 0.1.

To create and use a custom word database

  1. Create the database with the action CreateCustomWordDatabase. For example:

    /action=CreateCustomWordDatabase&database=words
  2. Add each custom word with the action TrainCustomSpeechWord. For example:

    /action=TrainCustomSpeechWord&database=words
                                 &word=AcmeSoft
                                 &baseword=Microsoft
                                 &weight=0.8
  3. When you configure your speech-to-text analysis task, use the configuration parameter CustomWordDatabase to specify the name of the custom word database that you created. For more information about configuring a speech-to-text analysis task, see Transcribe Speech.

For more information about the actions that you can use to manage custom word databases, refer to the Media Server Reference.