Changelog

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

3.11.3 (2024-09-03)

Note: Version bump only for package @crawlee/memory-storage

3.11.2 (2024-08-28)

Bug Fixes

• RequestQueueV2: remove inProgress cache, rely solely on locked states (#2601) (57fcb08)
• Use the correct mutex in memory storage RequestQueueClient (#2623) (2fa8a29)

3.11.1 (2024-07-24)

Note: Version bump only for package @crawlee/memory-storage

3.11.0 (2024-07-09)

Note: Version bump only for package @crawlee/memory-storage

3.10.5 (2024-06-12)

Note: Version bump only for package @crawlee/memory-storage

3.10.4 (2024-06-11)

Note: Version bump only for package @crawlee/memory-storage

3.10.3 (2024-06-07)

Note: Version bump only for package @crawlee/memory-storage

3.10.2 (2024-06-03)

Bug Fixes

• improve fix for double extension in KVS with HTML files (#2505) (157927d), closes #2419

3.10.1 (2024-05-23)

Note: Version bump only for package @crawlee/memory-storage

3.10.0 (2024-05-16)

Bug Fixes

• Fixed double extension for screenshots (#2419) (e8b39c4), closes #1980

3.9.2 (2024-04-17)

Note: Version bump only for package @crawlee/memory-storage

3.9.1 (2024-04-11)

Note: Version bump only for package @crawlee/memory-storage

3.9.0 (2024-04-10)

Note: Version bump only for package @crawlee/memory-storage

3.8.2 (2024-03-21)

Note: Version bump only for package @crawlee/memory-storage

3.8.1 (2024-02-22)

Note: Version bump only for package @crawlee/memory-storage

3.8.0 (2024-02-21)

Features

• KeyValueStore.recordExists() (#2339) (8507a65)

3.7.3 (2024-01-30)

Note: Version bump only for package @crawlee/memory-storage

3.7.2 (2024-01-09)

Note: Version bump only for package @crawlee/memory-storage

3.7.1 (2024-01-02)

Note: Version bump only for package @crawlee/memory-storage

3.7.0 (2023-12-21)

Bug Fixes

• MemoryStorage: lock request JSON file when reading to support multiple process crawling (#2215) (eb84ce9)

3.6.2 (2023-11-26)

Note: Version bump only for package @crawlee/memory-storage

3.6.1 (2023-11-15)

Note: Version bump only for package @crawlee/memory-storage

3.6.0 (2023-11-15)

Note: Version bump only for package @crawlee/memory-storage

3.5.8 (2023-10-17)

Bug Fixes

• MemoryStorage: ignore invalid files for request queues (#2132) (fa58581), closes #1985

3.5.7 (2023-10-05)

Note: Version bump only for package @crawlee/memory-storage

3.5.6 (2023-10-04)

Note: Version bump only for package @crawlee/memory-storage

3.5.5 (2023-10-02)

Features

• Request Queue v2 (#1975) (70a77ee), closes #1365

3.5.4 (2023-09-11)

Note: Version bump only for package @crawlee/memory-storage

3.5.3 (2023-08-31)

Bug Fixes

• pin all internal dependencies (#2041) (d6f2b17), closes #2040

3.5.2 (2023-08-21)

Note: Version bump only for package @crawlee/memory-storage

3.5.1 (2023-08-16)

Note: Version bump only for package @crawlee/memory-storage

3.5.0 (2023-07-31)

Bug Fixes

• cleanup worker stuff from memory storage to fix vitest (#2004) (d2e098c), closes #1999

3.4.2 (2023-07-19)

Note: Version bump only for package @crawlee/memory-storage

3.4.1 (2023-07-13)

Note: Version bump only for package @crawlee/memory-storage

3.4.0 (2023-06-12)

Note: Version bump only for package @crawlee/memory-storage

3.3.3 (2023-05-31)

Bug Fixes

• MemoryStorage: handle EXDEV errors when purging storages (#1932) (e656050)

3.3.2 (2023-05-11)

Bug Fixes

• MemoryStorage: cache requests in RequestQueue (#1899) (063dcd1)

Features

• RQv2 memory storage support (#1874) (049486b)

3.3.1 (2023-04-11)

Bug Fixes

• MemoryStorage: handling of readable streams for key-value stores when setting records (#1852) (a5ee37d), closes #1843

3.3.0 (2023-03-09)

Bug Fixes

• MemoryStorage: request queues race conditions causing crashes (#1806) (083a9db), closes #1792
• MemoryStorage: RequestQueue should respect forefront (#1816) (b68e86a), closes #1787
• MemoryStorage: RequestQueue#handledRequestCount should update (#1817) (a775e4a), closes #1764

Features

• add basic support for setStatusMessage (#1790) (c318980)
• move the status message implementation to Crawlee, noop in storage (#1808) (99c3fdc)

3.2.2 (2023-02-08)

Bug Fixes

• MemoryStorage: request queues saved in the wrong place (#1779) (19409db)

3.2.1 (2023-02-07)

Note: Version bump only for package @crawlee/memory-storage

3.2.0 (2023-02-07)

Bug Fixes

• Correctly compute pendingRequestCount in request queue (#1765) (946535f), closes /github.com/apify/crawlee/blob/master/packages/memory-storage/src/resource-clients/request-queue.ts#L291-L298
• KeyValueStore: big buffers should not crash (#1734) (2f682f7), closes #1732 #1710
• memory-storage: dont fail when storage already purged (#1737) (8694027), closes #1736
• utils: add missing dependency on ow (bf0e03c), closes #1716

Features

• MemoryStorage: read from fs if persistStorage is enabled, ram only otherwise (#1761) (e903980)

3.1.2 (2022-11-15)

Note: Version bump only for package @crawlee/memory-storage

3.1.1 (2022-11-07)

Note: Version bump only for package @crawlee/memory-storage

3.1.0 (2022-10-13)

Note: Version bump only for package @crawlee/memory-storage

3.0.4 (2022-08-22)

Bug Fixes

• key value stores emitting an error when multiple write promises ran in parallel (#1460) (f201cca)