Retrieve Information from a Filtered Stream

To automatically start the synchronize action and retrieve tweets from a filtered stream, create a new fetch task using the following procedure.

NOTE: Twitter allows a user account to connect to only one stream at a time. If you run multiple fetch tasks concurrently (by setting the TaskThreads parameter) and these tasks connect to Twitter streams, Twitter might close the connections.

To retrieve information from a filtered stream

  1. Stop the connector.
  2. Open the connector's configuration file in a text editor.
  3. In the [Ingestion] section of the configuration file, set the IngestBatchSize parameter:

    IngestBatchSize The number of documents to send to the ingestion server in each batch. If you are processing large numbers of streamed tweets and sending the information to CFS, Micro Focus recommends setting this parameter to 500 or greater. The Twitter Connector sends metadata-only documents to CFS, and CFS can process large batches more efficiently than small batches.
  4. In the [FetchTasks] section of the configuration file, specify the number of fetch tasks using the Number parameter. If you are configuring the first fetch task, type Number=1. If one or more fetch tasks have already been configured, increase the value of the Number parameter by one (1). Below the Number parameter, specify the names of the fetch tasks, starting from zero (0). For example:

    [FetchTasks]
    Number=1
    0=MyTask
  5. Below the [FetchTasks] section, create a new TaskName section. The name of the section must match the name of the new fetch task. For example:

    [FetchTasks]
    Number=1
    0=MyTask
       
    [MyTask]
  6. Ensure that the relevant OAuth parameters are included in the task. For information about how to configure authentication, see Configure OAuth Authentication.
  7. In the new section, set the following parameter:

    StreamType The type of stream to ingest tweets from. To retrieve tweets from a filtered stream, set this parameter to statuses/filter.
  8. Choose which tweets to synchronize.

    • Twitter V2 API.

      ApiVersion Set ApiVersion=V2.
      FilteredStreamRule

      A list of rules that specify the tweets to retrieve. You can specify multiple rules by using numbered parameters, for example FilteredStreamRule0, FilteredStreamRule1, FilteredStreamRule2, and so on. Twitter returns any tweets that match any of the rules.

      For information about the rule syntax and how to build rules, refer to the Twitter API documentation.

    • Twitter V1 API. The connector retrieves any tweet that matches Language (if set) and FilterLevel, and at least one of the parameters Follow, Location, or Track. You must set at least one of Follow, Location, or Track.

      ApiVersion Set ApiVersion=V1.
      Follow A comma-separated list of Twitter user names or IDs. Do not include the ‘@’ character before the Twitter user name.
      Location A comma-separated list of longitude and latitude values to identify specific geographic regions. Each region is a square, defined by four values in the following order: [Longitude West], [Latitude South], [Longitude East], [Latitude North].
      Track A comma-separated list of keywords.
      Language (Optional) To retrieve only those tweets that match specific languages, specify a comma-separated list of BCP 47 language codes that correspond to any of the languages listed on the Twitter advanced search page (https://twitter.com/search-advanced). For example, to retrieve only those tweets that Twitter has identified to be written in English, set Language=en.
      FilterLevel (Optional) To retrieve only those tweets that match or exceed a specific filter level, set the FilterLevel configuration parameter. For example, FilterLevel=medium.

    For more information about these parameters, refer to the Twitter Connector Reference.

  9. Save and close the configuration file. You can now start the connector.