UrlCantHaveRegex

A Perl-compatible regular expression to restrict the content retrieved by the connector. If the full URL of a page matches the regular expression, the page is not ingested. This parameter applies to all file types, for example HTML pages, images, text documents, and so on.

Type: String
Default:  
Required: No
Configuration Section: TaskName or FetchTasks or Default
Example: UrlCantHaveRegex=.*data\.mywebsite\.com/noindex.*|.*internal\.mywebsite\.com/noindex.*
See Also: UrlMustHaveRegex