Skip to main content
Version: 3.12

SitemapRequestListOptions

Hierarchy

  • UrlConstraints
    • SitemapRequestListOptions

Index

Properties

Properties

optionalexclude

exclude?: readonly (RegExp | GlobInput)[]

An array of glob pattern strings, regexp patterns or plain objects containing patterns matching URLs that will never be included.

The plain objects must include either the glob property or the regexp property.

Glob matching is always case-insensitive. If you need case-sensitive matching, provide a regexp.

optionalglobs

globs?: readonly GlobInput[]

An array of glob pattern strings or plain objects containing glob pattern strings matching the URLs to be enqueued.

The plain objects must include at least the glob property, which holds the glob pattern string.

The matching is always case-insensitive. If you need case-sensitive matching, use regexps property directly.

If globs is an empty array or undefined, and regexps are also not defined, then the SitemapRequestList includes all the URLs from the sitemap.

optionalmaxBufferSize

maxBufferSize?: number = 200

Maximum number of buffered URLs for the sitemap loading stream. If the buffer is full, the stream will pause until the buffer is drained.

optionalparseSitemapOptions

parseSitemapOptions?: Omit<ParseSitemapOptions, emitNestedSitemaps | maxDepth>

Advanced options for the underlying parseSitemap call.

optionalpersistStateKey

persistStateKey?: string

Key for persisting the state of the request list in the KeyValueStore.

optionalproxyUrl

proxyUrl?: string

Proxy URL to be used for sitemap loading.

optionalregexps

regexps?: readonly RegExpInput[]

An array of regular expressions or plain objects containing regular expressions matching the URLs to be enqueued.

The plain objects must include at least the regexp property, which holds the regular expression.

If regexps is an empty array or undefined, and globs are also not defined, then the SitemapRequestList includes all the URLs from the sitemap.

optionalsignal

signal?: AbortSignal

Abort signal to be used for sitemap loading.

sitemapUrls

sitemapUrls: string[]

List of sitemap URLs to parse.

optionaltimeoutMillis

timeoutMillis?: number

Timeout for sitemap loading in milliseconds. If both signal and timeoutMillis are provided, either of them can abort the loading.

Page Options