Skip to main content
Version: 3.1

RequestOptions <UserData>

Specifies required and optional fields for constructing a Request.




headers?: Record<string, string>

HTTP headers in the following format:

Accept: 'text/html',
'Content-Type': 'application/json'


keepUrlFragment?: boolean = false

If false then the hash part of a URL is removed when computing the uniqueKey property. For example, this causes the and URLs to have the same uniqueKey of and thus the URLs are considered equal. Note that this option only has an effect if uniqueKey is not set.


label?: string

Shortcut for setting userData: { label: '...' }.


method?: delete | AllowedHttpMethods | get | post | put | patch | head | trace | options | connect = delete | AllowedHttpMethods | get | post | put | patch | head | trace | options | connect


noRetry?: boolean = false

The true value indicates that the request will not be automatically retried on error.


payload?: string

HTTP request payload, e.g. for POST requests.


skipNavigation?: boolean = false

If set to true then the crawler processing this request evaluates the requestHandler immediately without prior browser navigation.


uniqueKey?: string

A unique key identifying the request. Two requests with the same uniqueKey are considered as pointing to the same URL.

If uniqueKey is not provided, then it is automatically generated by normalizing the URL. For example, the URL of HTTP:// will produce the uniqueKey of

The keepUrlFragment option determines whether URL hash fragment is included in the uniqueKey or not.

The useExtendedUniqueKey options determines whether method and payload are included in the uniqueKey, producing a uniqueKey in the following format: METHOD(payloadHash):normalizedUrl. This is useful when requests point to the same URL, but with different methods and payloads. For example: form submits.

Pass an arbitrary non-empty text value to the uniqueKey property to override the default behavior and specify which URLs shall be considered equal.


url: string

URL of the web page to crawl. It must be a non-empty string.


useExtendedUniqueKey?: boolean = false

If true then the uniqueKey is computed not only from the URL, but also from the method and payload properties. This is useful when making requests to the same URL that are differentiated by method or payload, such as form submit navigations in browsers.


userData?: UserData

Custom user data assigned to the request. Use this to save any request related data to the request's scope, keeping them accessible on retries, failures etc.