The MinPageDate
parameter filters the pages that are ingested by date. The connector only ingests pages that are newer than the specified date. Older pages are not ingested, though links from these pages might still be followed, depending on the value of SpiderDateFilteredPages.
Specify the date in any of the formats specified by DateFormats.
To filter pages by date the connector must be able to extract a date from the page URL, page content, or HTTP headers. Configure how to extract the date by setting DateInUrl, PageDateSelector, or PageDateHeader.
If you set the parameter MinPageAge, this parameter is ignored.
Type: | String |
Default: | |
Required: | No |
Configuration Section: | TaskName or FetchTasks or Default |
Example: | MinPageDate=2015-June-01
|
See Also: |
|