📄️ HTTP clients
Crawlee supports multiple HTTP clients when making requests.
📄️ HTTP crawlers
Crawlee supports multiple HTTP crawlers that can be used to extract data from server-rendered webpages.
📄️ Playwright crawler
How to use the PlaywrightCrawler and its related components.
📄️ Proxy management
Using proxies to get around those annoying IP-blocks
📄️ Request loaders
How to manage the requests your crawler will go through.
📄️ Scaling crawlers
Learn how to scale your crawlers by controlling concurrency and limiting requests per minute.
📄️ Session management
How to manage your cookies, proxy IP rotations and more.
📄️ Storages
How to work with storages in Crawlee, how to manage requests and how to store and retrieve scraping results.