Changelog

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.13.3 (2025-05-05)

Note: Version bump only for package @crawlee/browser

3.13.2 (2025-04-08)

Features

• add onSkippedRequest option (#2916) (764f992), closes #2910

3.13.1 (2025-04-07)

Bug Fixes

• don't double increment session usage count in BrowserCrawler (#2908) (3107e55), closes #2851
• rename RobotsFile to RobotsTxtFile (#2913) (3160f71), closes #2910

Features

• add respectRobotsTxtFile crawler option (#2910) (0eabed1)

3.13.0 (2025-03-04)

Note: Version bump only for package @crawlee/browser

3.12.2 (2025-01-27)

Note: Version bump only for package @crawlee/browser

3.12.1 (2024-12-04)

Note: Version bump only for package @crawlee/browser

3.12.0 (2024-11-04)

Note: Version bump only for package @crawlee/browser

3.11.5 (2024-10-04)

Bug Fixes

• puppeteer: rename ignoreHTTPSErrors to acceptInsecureCerts to support v23 (#2684) (f3927e6)

3.11.4 (2024-09-23)

Note: Version bump only for package @crawlee/browser

3.11.3 (2024-09-03)

Note: Version bump only for package @crawlee/browser

3.11.2 (2024-08-28)

Note: Version bump only for package @crawlee/browser

3.11.1 (2024-07-24)

Note: Version bump only for package @crawlee/browser

3.11.0 (2024-07-09)

Features

• add iframe expansion to parseWithCheerio in browsers (#2542) (328d085), closes #2507
• add ignoreIframes opt-out from the Cheerio iframe expansion (#2562) (474a8dc)

3.10.5 (2024-06-12)

Bug Fixes

• declare missing peer dependencies in @crawlee/browser package (#2532) (3357c7f)
• mark context.request.loadedUrl and id as required inside the request handler (#2531) (2b54660)

3.10.4 (2024-06-11)

Note: Version bump only for package @crawlee/browser

3.10.3 (2024-06-07)

Note: Version bump only for package @crawlee/browser

3.10.2 (2024-06-03)

Note: Version bump only for package @crawlee/browser

3.10.1 (2024-05-23)

Note: Version bump only for package @crawlee/browser

3.10.0 (2024-05-16)

Note: Version bump only for package @crawlee/browser

3.9.2 (2024-04-17)

Note: Version bump only for package @crawlee/browser

3.9.1 (2024-04-11)

Features

• browserPerProxy browser launch option (#2418) (df57b29)

3.9.0 (2024-04-10)

Features

• tieredProxyUrls for ProxyConfiguration (#2348) (5408c7f)
• better newUrlFunction for ProxyConfiguration (#2392) (330598b), closes #2348 #2065
• expand #shadow-root elements automatically in parseWithCheerio helper (#2396) (a05b3a9)

3.8.2 (2024-03-21)

Note: Version bump only for package @crawlee/browser

3.8.1 (2024-02-22)

Note: Version bump only for package @crawlee/browser

3.8.0 (2024-02-21)

Features

• adaptive playwright crawler (#2316) (8e4218a)

3.7.3 (2024-01-30)

Note: Version bump only for package @crawlee/browser

3.7.2 (2024-01-09)

Note: Version bump only for package @crawlee/browser

3.7.1 (2024-01-02)

Note: Version bump only for package @crawlee/browser

3.7.0 (2023-12-21)

Bug Fixes

• retryOnBlocked doesn't override the blocked HTTP codes (#2243) (81672c3)

Features

• check enqueue link strategy post redirect (#2238) (3c5f9d6), closes #2173
• log cause with retryOnBlocked (#2252) (e19a773), closes #2249

3.6.2 (2023-11-26)

Note: Version bump only for package @crawlee/browser

3.6.1 (2023-11-15)

Features

• puppeteer: enable new headless mode (#1910) (7fc999c)

3.6.0 (2023-11-15)

Note: Version bump only for package @crawlee/browser

3.5.8 (2023-10-17)

Note: Version bump only for package @crawlee/browser

3.5.7 (2023-10-05)

Note: Version bump only for package @crawlee/browser

3.5.6 (2023-10-04)

Note: Version bump only for package @crawlee/browser

3.5.5 (2023-10-02)

Features

• Request Queue v2 (#1975) (70a77ee), closes #1365

3.5.4 (2023-09-11)

Note: Version bump only for package @crawlee/browser

3.5.3 (2023-08-31)

Bug Fixes

• pin all internal dependencies (#2041) (d6f2b17), closes #2040

3.5.2 (2023-08-21)

Note: Version bump only for package @crawlee/browser

3.5.1 (2023-08-16)

Bug Fixes

• log original error message on session rotation (#2022) (8a11ffb)

3.5.0 (2023-07-31)

Features

• retire session on proxy error (#2002) (8c0928b), closes #1912

3.4.2 (2023-07-19)

Features

• retryOnBlocked detects blocked webpage (#1956) (766fa9b)

3.4.1 (2023-07-13)

Note: Version bump only for package @crawlee/browser

3.4.0 (2023-06-12)

Bug Fixes

• respect <base> when enqueuing (#1936) (aeef572)

3.3.3 (2023-05-31)

Note: Version bump only for package @crawlee/browser

3.3.2 (2023-05-11)

Note: Version bump only for package @crawlee/browser

3.3.1 (2023-04-11)

Note: Version bump only for package @crawlee/browser

3.3.0 (2023-03-09)

Bug Fixes

• ignore invalid URLs in enqueueLinks in browser crawlers (#1803) (5ac336c)

3.2.2 (2023-02-08)

Note: Version bump only for package @crawlee/browser

3.2.1 (2023-02-07)

Note: Version bump only for package @crawlee/browser

3.2.0 (2023-02-07)

Bug Fixes

• declare missing dependency on tslib (27e96c8), closes #1747

3.1.4 (2022-12-14)

Note: Version bump only for package @crawlee/browser

3.1.3 (2022-12-07)

Note: Version bump only for package @crawlee/browser

3.1.2 (2022-11-15)

Note: Version bump only for package @crawlee/browser

3.1.1 (2022-11-07)

Note: Version bump only for package @crawlee/browser

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/browser

3.0.4 (2022-08-22)

Features

• enable tab-as-a-container for Firefox (#1456) (ae5ba4f)