SpiderUrlMustHaveRegex

A Perl-compatible regular expression to restrict the pages that are crawled by the connector. The full URL of a page must match the regular expression, otherwise it is not crawled and is not ingested.

Type: String
Default:  
Required: No
Configuration Section: TaskName or FetchTasks or Default
Example: SpiderUrlMustHaveRegex=.*data\.mywebsite\.com.*|.*public\.mywebsite\.com.*
See Also: SpiderUrlCantHaveRegex

_HP_HTML5_bannerTitle.htm