Version: Next

Storage

Where crawlers keep data and requests: datasets, key-value stores, request queues, storage backends, and request loaders.

📄️Storages

How to work with storages in Crawlee, how to manage requests and how to store and retrieve scraping results.

How to work with storage clients in Crawlee, including the built-in clients and how to create your own.

How to manage the requests your crawler will go through.