The MaxPageDate
parameter filters the pages that are ingested by date. The connector only ingests pages that are older than the specified date. Newer pages are not ingested, though links from these pages might still be followed, depending on the value of SpiderDateFilteredPages.
Specify the date in any of the formats specified by DateFormats.
To filter pages by date the connector must be able to extract a date from the page URL, page content, or HTTP headers. Configure how to extract the date by setting DateInUrl, PageDateSelector, or PageDateHeader.
If you set the parameter MaxPageAge, this parameter is ignored.
Type: | String |
Default: | |
Required: | No |
Configuration Section: | TaskName or FetchTasks or Default |
Example: | MaxPageDate=2015-Feb-17
|
See Also: |
|