Skip to main content

HttpCrawlingContext

The crawling context used by the AbstractHttpCrawler.

Hierarchy

Index

Methods

from_basic_crawling_context

  • from_basic_crawling_context(*, context, http_response): Self
  • Convenience constructor that creates HttpCrawlingContext from existing BasicCrawlingContext.


    Parameters

    Returns Self

Properties

add_requests

add_requests: AddRequestsFunction

get_key_value_store

http_response

http_response: HttpResponse

The HTTP response received from the server.

log

log: logging.Logger

proxy_info

proxy_info: ProxyInfo | None

push_data

push_data: PushDataFunction

request

request: Request

send_request

send_request: SendRequestFunction

session

session: Session | None

use_state

use_state: UseStateFunction