Configure Generative Models in QMS

To use QMS to generate summaries, you must configure at least one generative model in the QMS configuration file.

You can configure QMS to use the following methods to generate embeddings:

  • a generative LLM. You create a generative LLM model by configuring QMS to use an available third-party model. QMS downloads and caches the model when you first use it.

  • a Lua or Python script model. You write a script that generates the summaries, for example by using a HTTP endpoint.

Choose a Generative Model

The generative models use LLMs from Hugging Face. You can choose generative text2text-generation models (see https://huggingface.co/models?pipeline_tag=text2text-generation).

OpenText recommends the google/flan-t5-small model (see https://huggingface.co/google/flan-t5-small).

Create a Lua Script Model

You can create your own Lua script model, using any method that you choose to generate the summaries. You can use the Lua model to interact or access third-party API summary generation.

The script must define a function called summarize. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

function summarize(text)
   return string.format("Here is your summary: %s, text")
end

Create a Python Script Model

You can create your own Python script model, using any method that you choose to generate the summaries. You can use the Python model to interact or access third-party API summary generation.

The script must define a function called summarize. This function must accept a single parameter, which is a string representing the text to generate summaries for. It must return the output summary as a string.

For example:

def summarize(text: str) -> str:
    '''
    Sample summarization function.
    '''
    return f'Here is your summary: {text}'

TIP: You can provide a requirements.txt file for any third-party modules that you want to use, by setting the RequirementsFile parameter in your model configuration. See Configure QMS with the Generative Model.

Configure QMS with the Generative Model

You configure embeddings in the [Generative] section of the QMS configuration file. See Generative Configuration Parameters.

To configure a generative model

  1. Open your configuration file in a text editor.

  2. Find the [Generative] section, or create one if it does not exist.

  3. Add an entry for your embedding model by using a zero-based index number. Set this value to the name of the configuration section to use to define your model. For example:

    [Generative]
    0=Summarizer
  4. Create a configuration section with the name that you defined.

    [Summarizer]
  5. Set the Type parameter to the type of model you want to use. For example:

    [Summarizer]
    Type=GenerativeLLM
  6. Set additional parameters for your model. The required parameters depend on the type of model:

  7. Save and close the configuration file.