ParselCrawlingContext
Hierarchy
- ParsedHttpCrawlingContext
- ParselCrawlingContext
Index
Methods
__hash__
Return hash of the context. Each context is considered unique.
Returns int
from_basic_crawling_context
Convenience constructor that creates
HttpCrawlingContextfrom existingBasicCrawlingContext.Parameters
context: BasicCrawlingContext
http_response: HttpResponse
Returns Self
from_http_crawling_context
Convenience constructor that creates new context from existing HttpCrawlingContext.
Parameters
context: HttpCrawlingContext
parsed_content: TParseResult
enqueue_links: EnqueueLinksFunction
Returns Self
from_parsed_http_crawling_context
Convenience constructor that creates new context from existing
ParsedHttpCrawlingContext[BeautifulSoup].Parameters
context: ParsedHttpCrawlingContext[Selector]
Returns Self
html_to_text
Convert the parsed HTML content to newline-separated plain text without tags.
Returns str
Properties
add_requests
Add requests crawling context helper function.
enqueue_links
get_key_value_store
Get key-value store crawling context helper function.
http_response
The HTTP response received from the server.
log
Logger instance.
parsed_content
proxy_info
Proxy information for the current page being processed.
push_data
Push data crawling context helper function.
request
Request object for the current page being processed.
selector
Convenience alias.
send_request
Send request crawling context helper function.
session
Session object for the current page being processed.
use_state
Use state crawling context helper function.
The crawling context used by the
ParselCrawler.It provides access to key objects as well as utility functions for handling crawling tasks.