HTTP clients
Learn about Crawlee's HTTP client architecture, how to switch between different implementations, and create custom HTTP clients for specialized web scraping needs.
HTTP headers
Learn how HTTP headers work in web scraping and how to work with them in Crawlee.
Session management
How to manage your cookies, proxy IP rotations and more.
Cookie management
How to read, set, and persist cookies across requests, retries, and runs in Crawlee.
Proxy management
How to use and rotate proxies in Crawlee - proxy configuration, IP rotation tied to sessions, and tiered proxies.