# crawlee<!-- -->

## Index[](#Index)

### Autoscaling

* [AutoscaledPool](https://crawlee.dev/python/python/api/class/AutoscaledPool.md)
* [Snapshotter](https://crawlee.dev/python/python/api/class/Snapshotter.md)
* [SystemStatus](https://crawlee.dev/python/python/api/class/SystemStatus.md)

### Browser management

* [BrowserController](https://crawlee.dev/python/python/api/class/BrowserController.md)
* [BrowserPlugin](https://crawlee.dev/python/python/api/class/BrowserPlugin.md)
* [BrowserPool](https://crawlee.dev/python/python/api/class/BrowserPool.md)
* [PlaywrightBrowserController](https://crawlee.dev/python/python/api/class/PlaywrightBrowserController.md)
* [PlaywrightBrowserPlugin](https://crawlee.dev/python/python/api/class/PlaywrightBrowserPlugin.md)
* [PlaywrightPersistentBrowser](https://crawlee.dev/python/python/api/class/PlaywrightPersistentBrowser.md)
* [StagehandBrowserController](https://crawlee.dev/python/python/api/class/StagehandBrowserController.md)
* [StagehandBrowserPlugin](https://crawlee.dev/python/python/api/class/StagehandBrowserPlugin.md)
* [StagehandOptions](https://crawlee.dev/python/python/api/class/StagehandOptions.md)
* [StagehandPage](https://crawlee.dev/python/python/api/class/StagehandPage.md)

### Configuration

* [ConcurrencySettings](https://crawlee.dev/python/python/api/class/ConcurrencySettings.md)
* [Configuration](https://crawlee.dev/python/python/api/class/Configuration.md)
* [ProxyConfiguration](https://crawlee.dev/python/python/api/class/ProxyConfiguration.md)
* [ServiceLocator](https://crawlee.dev/python/python/api/class/ServiceLocator.md)

### Crawlers

* [AbstractHttpCrawler](https://crawlee.dev/python/python/api/class/AbstractHttpCrawler.md)
* [AdaptivePlaywrightCrawler](https://crawlee.dev/python/python/api/class/AdaptivePlaywrightCrawler.md)
* [BasicCrawler](https://crawlee.dev/python/python/api/class/BasicCrawler.md)
* [BeautifulSoupCrawler](https://crawlee.dev/python/python/api/class/BeautifulSoupCrawler.md)
* [FileDownloadCrawler](https://crawlee.dev/python/python/api/class/FileDownloadCrawler.md)
* [HttpCrawler](https://crawlee.dev/python/python/api/class/HttpCrawler.md)
* [ParselCrawler](https://crawlee.dev/python/python/api/class/ParselCrawler.md)
* [PlaywrightCrawler](https://crawlee.dev/python/python/api/class/PlaywrightCrawler.md)
* [PydanticAiCrawler](https://crawlee.dev/python/python/api/class/PydanticAiCrawler.md)
* [StagehandCrawler](https://crawlee.dev/python/python/api/class/StagehandCrawler.md)

### Crawling contexts

* [AdaptivePlaywrightCrawlingContext](https://crawlee.dev/python/python/api/class/AdaptivePlaywrightCrawlingContext.md)
* [AdaptivePlaywrightPostNavCrawlingContext](https://crawlee.dev/python/python/api/class/AdaptivePlaywrightPostNavCrawlingContext.md)
* [AdaptivePlaywrightPreNavCrawlingContext](https://crawlee.dev/python/python/api/class/AdaptivePlaywrightPreNavCrawlingContext.md)
* [BasicCrawlingContext](https://crawlee.dev/python/python/api/class/BasicCrawlingContext.md)
* [BeautifulSoupCrawlingContext](https://crawlee.dev/python/python/api/class/BeautifulSoupCrawlingContext.md)
* [FileDownloadCrawlingContext](https://crawlee.dev/python/python/api/class/FileDownloadCrawlingContext.md)
* [HttpCrawlingContext](https://crawlee.dev/python/python/api/class/HttpCrawlingContext.md)
* [HttpCrawlingResult](https://crawlee.dev/python/python/api/class/HttpCrawlingResult.md)
* [ParsedHttpCrawlingContext](https://crawlee.dev/python/python/api/class/ParsedHttpCrawlingContext.md)
* [ParselCrawlingContext](https://crawlee.dev/python/python/api/class/ParselCrawlingContext.md)
* [PlaywrightCrawlingContext](https://crawlee.dev/python/python/api/class/PlaywrightCrawlingContext.md)
* [PlaywrightPostNavCrawlingContext](https://crawlee.dev/python/python/api/class/PlaywrightPostNavCrawlingContext.md)
* [PlaywrightPreNavCrawlingContext](https://crawlee.dev/python/python/api/class/PlaywrightPreNavCrawlingContext.md)
* [PydanticAiCrawlingContext](https://crawlee.dev/python/python/api/class/PydanticAiCrawlingContext.md)
* [StagehandCrawlingContext](https://crawlee.dev/python/python/api/class/StagehandCrawlingContext.md)
* [StagehandPostNavCrawlingContext](https://crawlee.dev/python/python/api/class/StagehandPostNavCrawlingContext.md)
* [StagehandPreNavCrawlingContext](https://crawlee.dev/python/python/api/class/StagehandPreNavCrawlingContext.md)

### Errors

* [ContextPipelineFinalizationError](https://crawlee.dev/python/python/api/class/ContextPipelineFinalizationError.md)
* [ContextPipelineInitializationError](https://crawlee.dev/python/python/api/class/ContextPipelineInitializationError.md)
* [ContextPipelineInterruptedError](https://crawlee.dev/python/python/api/class/ContextPipelineInterruptedError.md)
* [HttpClientStatusCodeError](https://crawlee.dev/python/python/api/class/HttpClientStatusCodeError.md)
* [HttpStatusCodeError](https://crawlee.dev/python/python/api/class/HttpStatusCodeError.md)
* [PersistentRateLimitError](https://crawlee.dev/python/python/api/class/PersistentRateLimitError.md)
* [ProxyError](https://crawlee.dev/python/python/api/class/ProxyError.md)
* [RequestCollisionError](https://crawlee.dev/python/python/api/class/RequestCollisionError.md)
* [RequestHandlerError](https://crawlee.dev/python/python/api/class/RequestHandlerError.md)
* [RequestThrottledError](https://crawlee.dev/python/python/api/class/RequestThrottledError.md)
* [ServiceConflictError](https://crawlee.dev/python/python/api/class/ServiceConflictError.md)
* [SessionError](https://crawlee.dev/python/python/api/class/SessionError.md)
* [UserDefinedErrorHandlerError](https://crawlee.dev/python/python/api/class/UserDefinedErrorHandlerError.md)

### Event data

* [Event](https://crawlee.dev/python/python/api/enum/Event.md)
* [EventAbortingData](https://crawlee.dev/python/python/api/class/EventAbortingData.md)
* [EventCrawlerStatusData](https://crawlee.dev/python/python/api/class/EventCrawlerStatusData.md)
* [EventExitData](https://crawlee.dev/python/python/api/class/EventExitData.md)
* [EventMigratingData](https://crawlee.dev/python/python/api/class/EventMigratingData.md)
* [EventPersistStateData](https://crawlee.dev/python/python/api/class/EventPersistStateData.md)
* [EventSystemInfoData](https://crawlee.dev/python/python/api/class/EventSystemInfoData.md)

### Event managers

* [EventManager](https://crawlee.dev/python/python/api/class/EventManager.md)
* [LocalEventManager](https://crawlee.dev/python/python/api/class/LocalEventManager.md)

### Functions

* [AddRequestsFunction](https://crawlee.dev/python/python/api/class/AddRequestsFunction.md)
* [BlockRequestsFunction](https://crawlee.dev/python/python/api/class/BlockRequestsFunction.md)
* [EnqueueLinksFunction](https://crawlee.dev/python/python/api/class/EnqueueLinksFunction.md)
* [ExtractFunction](https://crawlee.dev/python/python/api/class/ExtractFunction.md)
* [ExtractLinksFunction](https://crawlee.dev/python/python/api/class/ExtractLinksFunction.md)
* [GetKeyValueStoreFunction](https://crawlee.dev/python/python/api/class/GetKeyValueStoreFunction.md)
* [PushDataFunction](https://crawlee.dev/python/python/api/class/PushDataFunction.md)
* [PydanticAiHtmlDistiller](https://crawlee.dev/python/python/api/class/PydanticAiHtmlDistiller.md)
* [SendRequestFunction](https://crawlee.dev/python/python/api/class/SendRequestFunction.md)
* [UseStateFunction](https://crawlee.dev/python/python/api/class/UseStateFunction.md)

### HTTP clients

* [CurlImpersonateHttpClient](https://crawlee.dev/python/python/api/class/CurlImpersonateHttpClient.md)
* [HttpClient](https://crawlee.dev/python/python/api/class/HttpClient.md)
* [HttpxHttpClient](https://crawlee.dev/python/python/api/class/HttpxHttpClient.md)
* [ImpitHttpClient](https://crawlee.dev/python/python/api/class/ImpitHttpClient.md)

### HTTP parsers

* [AbstractHttpParser](https://crawlee.dev/python/python/api/class/AbstractHttpParser.md)
* [BeautifulSoupParser](https://crawlee.dev/python/python/api/class/BeautifulSoupParser.md)
* [NoParser](https://crawlee.dev/python/python/api/class/NoParser.md)
* [ParselParser](https://crawlee.dev/python/python/api/class/ParselParser.md)

### Other

* [BasePydanticAiHtmlDistiller](https://crawlee.dev/python/python/api/class/BasePydanticAiHtmlDistiller.md)
* [BasePydanticAiHtmlExtractor](https://crawlee.dev/python/python/api/class/BasePydanticAiHtmlExtractor.md)
* [BrowserforgeFingerprintGenerator](https://crawlee.dev/python/python/api/class/BrowserforgeFingerprintGenerator.md)
* [ContextPipeline](https://crawlee.dev/python/python/api/class/ContextPipeline.md)
* [CrawlerInstrumentor](https://crawlee.dev/python/python/api/class/CrawlerInstrumentor.md)
* [DefaultRenderingTypePredictor](https://crawlee.dev/python/python/api/class/DefaultRenderingTypePredictor.md)
* [FingerprintGenerator](https://crawlee.dev/python/python/api/class/FingerprintGenerator.md)
* [HeaderGenerator](https://crawlee.dev/python/python/api/class/HeaderGenerator.md)
* [HttpHeaders](https://crawlee.dev/python/python/api/class/HttpHeaders.md)
* [HttpResponse](https://crawlee.dev/python/python/api/class/HttpResponse.md)
* [PageSnapshot](https://crawlee.dev/python/python/api/class/PageSnapshot.md)
* [ProxyInfo](https://crawlee.dev/python/python/api/class/ProxyInfo.md)
* [PydanticAiCleanHtmlDistiller](https://crawlee.dev/python/python/api/class/PydanticAiCleanHtmlDistiller.md)
* [PydanticAiDirectExtractor](https://crawlee.dev/python/python/api/class/PydanticAiDirectExtractor.md)
* [PydanticAiHtmlExtractor](https://crawlee.dev/python/python/api/class/PydanticAiHtmlExtractor.md)
* [PydanticAiSelectorExtractor](https://crawlee.dev/python/python/api/class/PydanticAiSelectorExtractor.md)
* [PydanticAiSkeletonDistiller](https://crawlee.dev/python/python/api/class/PydanticAiSkeletonDistiller.md)
* [PydanticAiUsageStats](https://crawlee.dev/python/python/api/class/PydanticAiUsageStats.md)
* [RenderingTypePrediction](https://crawlee.dev/python/python/api/class/RenderingTypePrediction.md)
* [RenderingTypePredictor](https://crawlee.dev/python/python/api/class/RenderingTypePredictor.md)
* [RequestOptions](https://crawlee.dev/python/python/api/class/RequestOptions.md)
* [Router](https://crawlee.dev/python/python/api/class/Router.md)

### Request loaders

* [RequestList](https://crawlee.dev/python/python/api/class/RequestList.md)
* [RequestLoader](https://crawlee.dev/python/python/api/class/RequestLoader.md)
* [RequestManager](https://crawlee.dev/python/python/api/class/RequestManager.md)
* [RequestManagerTandem](https://crawlee.dev/python/python/api/class/RequestManagerTandem.md)
* [SitemapRequestLoader](https://crawlee.dev/python/python/api/class/SitemapRequestLoader.md)
* [ThrottlingRequestManager](https://crawlee.dev/python/python/api/class/ThrottlingRequestManager.md)

### Session management

* [CookieParam](https://crawlee.dev/python/python/api/class/CookieParam.md)
* [Session](https://crawlee.dev/python/python/api/class/Session.md)
* [SessionCookies](https://crawlee.dev/python/python/api/class/SessionCookies.md)
* [SessionPool](https://crawlee.dev/python/python/api/class/SessionPool.md)

### Statistics

* [AdaptivePlaywrightCrawlerStatisticState](https://crawlee.dev/python/python/api/class/AdaptivePlaywrightCrawlerStatisticState.md)
* [FinalStatistics](https://crawlee.dev/python/python/api/class/FinalStatistics.md)
* [Statistics](https://crawlee.dev/python/python/api/class/Statistics.md)
* [StatisticsState](https://crawlee.dev/python/python/api/class/StatisticsState.md)

### Storage clients

* [FileSystemStorageClient](https://crawlee.dev/python/python/api/class/FileSystemStorageClient.md)
* [MemoryStorageClient](https://crawlee.dev/python/python/api/class/MemoryStorageClient.md)
* [RedisStorageClient](https://crawlee.dev/python/python/api/class/RedisStorageClient.md)
* [SqlStorageClient](https://crawlee.dev/python/python/api/class/SqlStorageClient.md)
* [StorageClient](https://crawlee.dev/python/python/api/class/StorageClient.md)

### Storage data

* [AddRequestsResponse](https://crawlee.dev/python/python/api/class/AddRequestsResponse.md)
* [DatasetItemsListPage](https://crawlee.dev/python/python/api/class/DatasetItemsListPage.md)
* [DatasetMetadata](https://crawlee.dev/python/python/api/class/DatasetMetadata.md)
* [KeyValueStoreMetadata](https://crawlee.dev/python/python/api/class/KeyValueStoreMetadata.md)
* [KeyValueStoreRecord](https://crawlee.dev/python/python/api/class/KeyValueStoreRecord.md)
* [KeyValueStoreRecordMetadata](https://crawlee.dev/python/python/api/class/KeyValueStoreRecordMetadata.md)
* [ProcessedRequest](https://crawlee.dev/python/python/api/class/ProcessedRequest.md)
* [Request](https://crawlee.dev/python/python/api/class/Request.md)
* [RequestQueueMetadata](https://crawlee.dev/python/python/api/class/RequestQueueMetadata.md)
* [StorageMetadata](https://crawlee.dev/python/python/api/class/StorageMetadata.md)
* [UnprocessedRequest](https://crawlee.dev/python/python/api/class/UnprocessedRequest.md)

### Storages

* [Dataset](https://crawlee.dev/python/python/api/class/Dataset.md)
* [KeyValueStore](https://crawlee.dev/python/python/api/class/KeyValueStore.md)
* [RequestQueue](https://crawlee.dev/python/python/api/class/RequestQueue.md)
* [Storage](https://crawlee.dev/python/python/api/class/Storage.md)