@crawlee/utils

Index

References

• ArgumentValidationError
• EnqueueStrategy

Namespaces

• social

Classes

• RobotsTxtFile
• Sitemap

Interfaces

• DownloadListOfUrlsOptions
• ExtractUrlsOptions
• OpenGraphProperty
• ParseSitemapOptions
• RobotsTxtFileSitemapsOptions

Type Aliases

• SitemapUrl

Functions

• discoverValidSitemaps
• downloadListOfUrls
• extractUrls
• htmlToText
• parseOpenGraph
• parseSitemap
• sleep

References

ArgumentValidationError

Re-exports ArgumentValidationError

EnqueueStrategy

Re-exports EnqueueStrategy

Type Aliases

SitemapUrl

SitemapUrl: SitemapUrlData & { originSitemapUrl: string }