Skip to main content

AddRequestsFunction

Type of a function for adding URLs to the request queue with optional filtering.

This helper method simplifies the process of adding requests to the request provider. It opens the specified request provider and adds the requests to it.

Index

Methods

Methods

__call__

  • __call__(*, requests, kwargs): Coroutine[None, None, None]

  • Parameters

    • optionalkeyword-onlyrequests: Sequence[(str | BaseRequestData) | Request]
    • keyword-onlylimit: int

      Maximum number of requests to be enqueued.

    • keyword-onlybase_url: str

      Base URL to be used for relative URLs.

    • keyword-onlystrategy: EnqueueStrategy

      Enqueueing strategy, see the EnqueueStrategy enum for possible values and their meanings.

    • keyword-onlyinclude: list[re.Pattern | Glob]

      List of regular expressions or globs that URLs must match to be enqueued.

    • keyword-onlyexclude: list[re.Pattern | Glob]

      List of regular expressions or globs that URLs must not match to be enqueued.

    Returns Coroutine[None, None, None]

Page Options