Skip to main content
Version: 3.0

Changelog

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.13.3 (2025-05-05)​

Note: Version bump only for package @crawlee/utils

3.13.2 (2025-04-08)​

Note: Version bump only for package @crawlee/utils

3.13.1 (2025-04-07)​

Bug Fixes​

Features​

3.13.0 (2025-03-04)

Features​

3.12.2 (2025-01-27)​

Note: Version bump only for package @crawlee/utils

3.12.1 (2024-12-04)​

Bug Fixes​

  • social: support new URL formats for Facebook, YouTube and X (#2758) (4c95847), closes #525

3.12.0 (2024-11-04)

Bug Fixes​

Features​

3.11.5 (2024-10-04)​

Note: Version bump only for package @crawlee/utils

3.11.4 (2024-09-23)​

Bug Fixes​

3.11.3 (2024-09-03)​

Bug Fixes​

  • improve FACEBOOK_REGEX to match older style page URLs (#2650) (a005e69), closes #2216

3.11.2 (2024-08-28)​

Bug Fixes​

  • use namespace imports for cheerio to be compatible with v1 (#2641) (f48296f)

Features​

3.11.1 (2024-07-24)​

Bug Fixes​

3.11.0 (2024-07-09)

Features​

  • Sitemap-based request list implementation (#2498) (7bf8f0b)

3.10.5 (2024-06-12)​

Note: Version bump only for package @crawlee/utils

3.10.4 (2024-06-11)​

Note: Version bump only for package @crawlee/utils

3.10.3 (2024-06-07)​

Bug Fixes​

  • respect implicit router when no requestHandler is provided in AdaptiveCrawler (#2518) (31083aa)

3.10.2 (2024-06-03)​

Bug Fixes​

Features​

3.10.1 (2024-05-23)​

Bug Fixes​

  • adjust URL_NO_COMMAS_REGEX regexp to allow single character hostnames (#2492) (ec802e8), closes #2487

3.10.0 (2024-05-16)

Bug Fixes​

  • malformed sitemap url when sitemap index child contains querystring (#2430) (e4cd41c)
  • return true when robots.isAllowed returns undefined (#2439) (6f541f8), closes #2437
  • sitemap content-type check breaks on content-type parameters (#2442) (db7d372)

Features​

  • implement ErrorSnapshotter for error context capture (#2332) (e861dfd), closes #2280

3.9.2 (2024-04-17)​

Features​

3.9.1 (2024-04-11)​

Note: Version bump only for package @crawlee/utils

3.9.0 (2024-04-10)

Bug Fixes​

Features​

  • expand #shadow-root elements automatically in parseWithCheerio helper (#2396) (a05b3a9)

3.8.2 (2024-03-21)​

Bug Fixes​

  • correctly report gzip decompression errors (#2368) (84a2f17)

3.8.1 (2024-02-22)​

Note: Version bump only for package @crawlee/utils

3.8.0 (2024-02-21)

Features​

  • add Sitemap.tryCommonNames to check well known sitemap locations (#2311) (85589f1), closes #2307
  • core: add userAgent parameter to RobotsFile.isAllowed() + RobotsFile.from() helper (#2338) (343c159)
  • Support plain-text sitemap files (sitemap.txt) (#2315) (0bee7da)

3.7.3 (2024-01-30)​

Bug Fixes​

3.7.2 (2024-01-09)​

Note: Version bump only for package @crawlee/utils

3.7.1 (2024-01-02)​

Bug Fixes​

  • ES2022 build compatibility and move to NodeNext for module (#2258) (7fe1e68), closes #2257

3.7.0 (2023-12-21)

Bug Fixes​

  • retryOnBlocked doesn't override the blocked HTTP codes (#2243) (81672c3)

Features​

3.6.2 (2023-11-26)​

Note: Version bump only for package @crawlee/utils

3.6.1 (2023-11-15)​

Note: Version bump only for package @crawlee/utils

3.6.0 (2023-11-15)

Features​

3.5.8 (2023-10-17)​

Bug Fixes​

  • refactor extractUrls to split the text line by line first (#2122) (7265cd7)

3.5.7 (2023-10-05)​

Note: Version bump only for package @crawlee/utils

3.5.6 (2023-10-04)​

Features​

3.5.5 (2023-10-02)​

Note: Version bump only for package @crawlee/utils

3.5.4 (2023-09-11)​

Note: Version bump only for package @crawlee/utils

3.5.3 (2023-08-31)​

Bug Fixes​

3.5.2 (2023-08-21)​

Note: Version bump only for package @crawlee/utils

3.5.1 (2023-08-16)​

Note: Version bump only for package @crawlee/utils

3.5.0 (2023-07-31)

Features​

3.4.2 (2023-07-19)​

Features​

3.4.1 (2023-07-13)​

Note: Version bump only for package @crawlee/utils

3.4.0 (2023-06-12)

Note: Version bump only for package @crawlee/utils

3.3.3 (2023-05-31)​

Note: Version bump only for package @crawlee/utils

3.3.2 (2023-05-11)​

Note: Version bump only for package @crawlee/utils

3.3.1 (2023-04-11)​

Bug Fixes​

  • jsdom: delay closing of the window and add some polyfills (2e81618)

3.3.0 (2023-03-09)

Bug Fixes​

  • add proxyUrl to DownloadListOfUrlsOptions (779be1e), closes #1780

3.2.2 (2023-02-08)​

Note: Version bump only for package @crawlee/utils

3.2.1 (2023-02-07)​

Note: Version bump only for package @crawlee/utils

3.2.0 (2023-02-07)

Bug Fixes​

  • utils: add missing dependency on ow (bf0e03c), closes #1716

3.1.2 (2022-11-15)​

Note: Version bump only for package @crawlee/utils

3.1.1 (2022-11-07)​

Note: Version bump only for package @crawlee/utils

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/utils

3.0.4 (2022-08-22)​

Note: Version bump only for package @crawlee/utils

Page Options