PlaywrightHttpClient

HTTP client based on the Playwright library.

This client uses the Playwright library to perform HTTP requests in crawlers (BasicCrawler subclasses) and to manage sessions, proxies, and error handling.

See the HttpClient class for more common information about HTTP clients.

Note: This class is pre-designated for use in PlaywrightCrawler only

Hierarchy

• HttpClient
  • PlaywrightHttpClient

Index

Methods

• __init__
• crawl
• send_request

Methods

__init__

• __init__(): None

• Overrides HttpClient.__init__

  Initialize a new instance.

  ---

  Returns None

crawl

• async crawl(request, *, session, proxy_info, statistics): HttpCrawlingResult

• Overrides HttpClient.crawl

  Perform the crawling for a given request.

  This method is called from crawler.run().

  ---

  Parameters

  • request: Request

    The request to be crawled.

  • optionalkeyword-onlysession: Session | None = None

    The session associated with the request.

  • optionalkeyword-onlyproxy_info: ProxyInfo | None = None

    The information about the proxy to be used.

  • optionalkeyword-onlystatistics: Statistics | None = None

    The statistics object to register status codes.

  Returns HttpCrawlingResult

send_request

• async send_request(url, *, method, headers, payload, session, proxy_info): HttpResponse

• Overrides HttpClient.send_request

  Send an HTTP request via the client.

  This method is called from context.send_request() helper.

  ---

  Parameters

  • url: str

    The URL to send the request to.

  • optionalkeyword-onlymethod: Literal[GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH] = 'GET'

    The HTTP method to use.

  • optionalkeyword-onlyheaders: (HttpHeaders | dict[str, str]) | None = None

    The headers to include in the request.

  • optionalkeyword-onlypayload: bytes | None = None

    The data to be sent as the request body.

  • optionalkeyword-onlysession: Session | None = None

    The session associated with the request.

  • optionalkeyword-onlyproxy_info: ProxyInfo | None = None

    The information about the proxy to be used.

  Returns HttpResponse