Group Messages

The Slack Connector creates documents that represent messages, but messages are likely to contain relatively little content. You might prefer to have documents that contain several messages, so that searches and other IDOL operations return part of a conversation instead of a single question or response.

You can use the configuration parameters GroupMessagesByCount and GroupMessagesByInterval to configure the connector so that it groups messages and creates a document for each message group.

  • GroupMessagesByCount configures the connector to group a specific number of messages. For example, if you set GroupMessagesByCount=5 the connector creates documents that contain up to five messages.
  • GroupMessagesByInterval configures the connector to group messages that occur within a specific time period. If you set GroupMessagesByInterval=5 minutes, the connector groups messages that occur within a five-minute interval. The next interval begins at the time of the first message that falls outside the preceding interval. For example, notice the one minute gap between the following five-minute intervals:

    10:30 - Message1      [Interval 1 starts at 10:30]
    10:32 - Message2      [In interval 1]
    10:34 - Message3      [In interval 1]
                          [Interval 1 ends at 10:35]
    10:36 - Message4      [Interval 2 starts at 10:36]
    10:38 - Message5      [In interval 2]
    10:40 - Message6      [In interval 2]

If you set both of these parameters, the connector starts a new document as soon as one of the criteria (time interval or number of messages) is reached.

The following is an example document created by the Slack connector, that represents a single message:

<DOCUMENT>
   <DREREFERENCE>...</DREREFERENCE>
   <AUTN_GROUP>Connector</AUTN_GROUP>
   <AUTN_IDENTIFIER>...</AUTN_IDENTIFIER>
   <CHANNEL_ID>D3E4P84QM</CHANNEL_ID>
   <CHANNEL_TYPE>IM</CHANNEL_TYPE>
   <DOC_TYPE>Message</DOC_TYPE>
   <IS_HIDDEN>false</IS_HIDDEN>
   <IS_STARRED>false</IS_STARRED>
   <MESSAGE_SUBTYPE/>
   <MESSAGE_TIMESTAMP>2016-Dec-20 16:27:39 GMT Daylight Time</MESSAGE_TIMESTAMP>
   <MESSAGE_TIMESTAMP_SEQ_ID>000013</MESSAGE_TIMESTAMP_SEQ_ID>
   <MESSAGE_TYPE>message</MESSAGE_TYPE>
   <SECURITYTYPE>SLACK</SECURITYTYPE>
   <SLACK_METADATA/>
   <TEAM_ID>T3E20B2Q2</TEAM_ID>
   <TEAM_NAME>IDOL Connectors</TEAM_NAME>
   <TEAM_URL>https://idolconnectors.slack.com/</TEAM_URL>
   <USER_ID>USLACKBOT</USER_ID>
   <USERNAME>slackbot</USERNAME>
   <DRECONTENT>
          I searched for that on our Help Center. Perhaps these articles will help: 
          - Change your time zone
          - Manage your password
   </DRECONTENT>
</DOCUMENT>

The following is an example document that contains three messages. Each MESSAGE field contains metadata for a message, and there are three pages of content that contain the message text. These fields are ordered so that the first MESSAGE metadata field corresponds to the first page of content, and so on.

<DOCUMENT>
   <DREREFERENCE>...</DREREFERENCE>
   <AUTN_GROUP>Connector</AUTN_GROUP>
   <AUTN_IDENTIFIER>...</AUTN_IDENTIFIER>
   <CHANNEL_ID>D3E4P84QM</CHANNEL_ID>
   <CHANNEL_TYPE>IM</CHANNEL_TYPE>
   <DOC_TYPE>Message</DOC_TYPE>
   <MESSAGE>
      <IS_HIDDEN>false</IS_HIDDEN>
      <IS_STARRED>false</IS_STARRED>
      <MESSAGE_SUBTYPE/>
      <MESSAGE_TIMESTAMP>2016-Dec-20 16:26:59 GMT Daylight Time</MESSAGE_TIMESTAMP>
      <MESSAGE_TIMESTAMP_SEQ_ID>000008</MESSAGE_TIMESTAMP_SEQ_ID>
      <MESSAGE_TYPE>message</MESSAGE_TYPE>
      <SLACK_METADATA/>
      <USER_ID>U3ENBA37Z</USER_ID>
      <USERNAME>...</USERNAME>
   </MESSAGE>
   <MESSAGE>
      <IS_HIDDEN>false</IS_HIDDEN>
      <IS_STARRED>false</IS_STARRED>
      <MESSAGE_SUBTYPE/>
      <MESSAGE_TIMESTAMP>2016-Dec-20 16:26:59 GMT Daylight Time</MESSAGE_TIMESTAMP>
      <MESSAGE_TIMESTAMP_SEQ_ID>000009</MESSAGE_TIMESTAMP_SEQ_ID>
      <MESSAGE_TYPE>message</MESSAGE_TYPE>
      <SLACK_METADATA/>
      <USER_ID>USLACKBOT</USER_ID>
      <USERNAME>slackbot</USERNAME>
   </MESSAGE>
   <MESSAGE>
      <IS_HIDDEN>false</IS_HIDDEN>
      <IS_STARRED>false</IS_STARRED>
      <MESSAGE_SUBTYPE/>
      <MESSAGE_TIMESTAMP>2016-Dec-20 16:27:18 GMT Daylight Time</MESSAGE_TIMESTAMP>
      <MESSAGE_TIMESTAMP_SEQ_ID>000010</MESSAGE_TIMESTAMP_SEQ_ID>
      <MESSAGE_TYPE>message</MESSAGE_TYPE>
      <SLACK_METADATA/>
      <USER_ID>U3ENBA37Z</USER_ID>
      <USERNAME>...</USERNAME>
   </MESSAGE>
   <MESSAGE_COUNT>3</MESSAGE_COUNT>
   <SECURITYTYPE>SLACK</SECURITYTYPE>
   <SLACK_METADATA/>
   <TEAM_ID>T3E20B2Q2</TEAM_ID>
   <TEAM_NAME>IDOL Connectors</TEAM_NAME>
   <TEAM_URL>https://idolconnectors.slack.com/</TEAM_URL>
   <DRECONTENT>message1</DRECONTENT>
   <DRECONTENT>message2</DRECONTENT>
   <DRECONTENT>message3</DRECONTENT>
</DOCUMENT>