PydanticAiCrawlingContext
Hierarchy
- ParselCrawlingContext
- PydanticAiCrawlingContext
Index
Methods
Properties
Methods
__hash__
Return hash of the context. Each context is considered unique.
Returns int
create_modified_copy
Create a modified copy of the crawling context with specified changes.
Parameters
optionalpush_data: PushDataFunction | None = None
optionaladd_requests: AddRequestsFunction | None = None
optionalget_key_value_store: GetKeyValueStoreFromRequestHandlerFunction | None = None
Returns Self
from_basic_crawling_context
Initialize a new instance from an existing
BasicCrawlingContext.Parameters
context: BasicCrawlingContext
http_response: HttpResponse
Returns Self
from_http_crawling_context
Initialize a new instance from an existing
HttpCrawlingContext.Parameters
context: HttpCrawlingContext
parsed_content: TParseResult
enqueue_links: EnqueueLinksFunction
extract_links: ExtractLinksFunction
Returns Self
from_parsed_http_crawling_context
Create a new context from an existing
ParsedHttpCrawlingContext[Selector].Parameters
context: ParsedHttpCrawlingContext[Selector]
Returns Self
from_parsel_crawling_context
Create a new context from an existing
ParselCrawlingContext.Parameters
context: ParselCrawlingContext
keyword-onlyextract: ExtractFunction
keyword-onlyai_usage: PydanticAiUsageStats
Returns Self
get_snapshot
Get snapshot of crawled page.
Returns PageSnapshot
html_to_text
Convert the parsed HTML content to newline-separated plain text without tags.
Returns str
Properties
add_requests
Add requests crawling context helper function.
ai_usage
The cumulative token usage stats of the extractor across calls in this crawl.
enqueue_links
extract
Extract a structured Pydantic model from the page using the configured AI extractor.
extract_links
get_key_value_store
Get key-value store crawling context helper function.
http_response
The HTTP response received from the server.
log
Logger instance.
parsed_content
proxy_info
Proxy information for the current page being processed.
push_data
Push data crawling context helper function.
register_deferred_cleanup
Register an async callback to be called after request processing completes (including error handlers).
request
Request object for the current page being processed.
selector
Convenience alias.
send_request
Send request crawling context helper function.
session
Session object for the current page being processed.
use_state
Use state crawling context helper function.
The crawling context used by the
PydanticAiCrawler.It extends
ParselCrawlingContext, so the full Parselselector(andenqueue_links) remain available alongside the AI-poweredextracthelper. Handlers can mix cheap manual selectors with AI extraction on the same page.