Skip to main content
Version: 3.8

Changelog

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.8.1 (2024-02-22)

Note: Version bump only for package @crawlee/utils

3.8.0 (2024-02-21)

Features

  • add Sitemap.tryCommonNames to check well known sitemap locations (#2311) (85589f1), closes #2307
  • core: add userAgent parameter to RobotsFile.isAllowed() + RobotsFile.from() helper (#2338) (343c159)
  • Support plain-text sitemap files (sitemap.txt) (#2315) (0bee7da)

3.7.3 (2024-01-30)

Bug Fixes

3.7.2 (2024-01-09)

Note: Version bump only for package @crawlee/utils

3.7.1 (2024-01-02)

Bug Fixes

  • ES2022 build compatibility and move to NodeNext for module (#2258) (7fe1e68), closes #2257

3.7.0 (2023-12-21)

Bug Fixes

  • retryOnBlocked doesn't override the blocked HTTP codes (#2243) (81672c3)

Features

3.6.2 (2023-11-26)

Note: Version bump only for package @crawlee/utils

3.6.1 (2023-11-15)

Note: Version bump only for package @crawlee/utils

3.6.0 (2023-11-15)

Features

3.5.8 (2023-10-17)

Bug Fixes

  • refactor extractUrls to split the text line by line first (#2122) (7265cd7)

3.5.7 (2023-10-05)

Note: Version bump only for package @crawlee/utils

3.5.6 (2023-10-04)

Features

3.5.5 (2023-10-02)

Note: Version bump only for package @crawlee/utils

3.5.4 (2023-09-11)

Note: Version bump only for package @crawlee/utils

3.5.3 (2023-08-31)

Bug Fixes

3.5.2 (2023-08-21)

Note: Version bump only for package @crawlee/utils

3.5.1 (2023-08-16)

Note: Version bump only for package @crawlee/utils

3.5.0 (2023-07-31)

Features

3.4.2 (2023-07-19)

Features

3.4.1 (2023-07-13)

Note: Version bump only for package @crawlee/utils

3.4.0 (2023-06-12)

Note: Version bump only for package @crawlee/utils

3.3.3 (2023-05-31)

Note: Version bump only for package @crawlee/utils

3.3.2 (2023-05-11)

Note: Version bump only for package @crawlee/utils

3.3.1 (2023-04-11)

Bug Fixes

  • jsdom: delay closing of the window and add some polyfills (2e81618)

3.3.0 (2023-03-09)

Bug Fixes

  • add proxyUrl to DownloadListOfUrlsOptions (779be1e), closes #1780

3.2.2 (2023-02-08)

Note: Version bump only for package @crawlee/utils

3.2.1 (2023-02-07)

Note: Version bump only for package @crawlee/utils

3.2.0 (2023-02-07)

Bug Fixes

  • utils: add missing dependency on ow (bf0e03c), closes #1716

3.1.2 (2022-11-15)

Note: Version bump only for package @crawlee/utils

3.1.1 (2022-11-07)

Note: Version bump only for package @crawlee/utils

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/utils

3.0.4 (2022-08-22)

Note: Version bump only for package @crawlee/utils