CrawlerRunOptions

Hierarchy

• CrawlerAddRequestsOptions
  • CrawlerRunOptions

Index

Properties

• forefront
• purgeRequestQueue
• waitForAllRequestsToBeAdded

Properties

optionalforefront

forefront?: boolean = <p>false</p>

Inherited from CrawlerAddRequestsOptions.forefront

If set to true:

• while adding the request to the queue: the request will be added to the foremost position in the queue.
• while reclaiming the request: the request will be placed to the beginning of the queue, so that it's returned in the next call to RequestQueue.fetchNextRequest. By default, it's put to the end of the queue.

optionalpurgeRequestQueue

purgeRequestQueue?: boolean = <p>true</p>

Whether to purge the RequestQueue before running the crawler again. Defaults to true, so it is possible to reprocess failed requests. When disabled, only new requests will be considered. Note that even a failed request is considered as handled.

optionalwaitForAllRequestsToBeAdded

waitForAllRequestsToBeAdded?: boolean = <p>false</p>

Inherited from CrawlerAddRequestsOptions.waitForAllRequestsToBeAdded

Whether to wait for all the provided requests to be added, instead of waiting just for the initial batch of up to 1000.