Skip to main content

RequestQueueClient

crawlee.memory_storage_client._request_queue_client.RequestQueueClient

Subclient for manipulating a single request queue.

Index

Constructors

__init__

  • __init__(*, memory_storage_client, id, name, created_at, accessed_at, modified_at, handled_request_count, pending_request_count): None
  • Parameters

    • memory_storage_client: MemoryStorageClientkeyword-only
    • id: str | None = Nonekeyword-only
    • name: str | None = Nonekeyword-only
    • created_at: datetime | None = Nonekeyword-only
    • accessed_at: datetime | None = Nonekeyword-only
    • modified_at: datetime | None = Nonekeyword-only
    • handled_request_count: int = 0keyword-only
    • pending_request_count: int = 0keyword-only

    Returns None

Methods

add_request

  • async add_request(request, *, forefront): ProcessedRequest
  • Parameters

    • request: Request
    • forefront: bool = Falsekeyword-only

    Returns ProcessedRequest

batch_add_requests

  • async batch_add_requests(requests, *, forefront): BatchRequestsOperationResponse
  • Parameters

    • requests: Sequence[Request]
    • forefront: bool = Falsekeyword-only

    Returns BatchRequestsOperationResponse

batch_delete_requests

  • async batch_delete_requests(requests): BatchRequestsOperationResponse
  • Parameters

    • requests: list[Request]

    Returns BatchRequestsOperationResponse

delete

  • async delete(): None
  • Returns None

delete_request

  • async delete_request(request_id): None
  • Parameters

    • request_id: str

    Returns None

delete_request_lock

  • async delete_request_lock(request_id, *, forefront): None
  • Parameters

    • request_id: str
    • forefront: bool = Falsekeyword-only

    Returns None

get

  • async get(): RequestQueueMetadata | None
  • Returns RequestQueueMetadata | None

get_request

  • async get_request(request_id): Request | None
  • Parameters

    • request_id: str

    Returns Request | None

list_and_lock_head

  • async list_and_lock_head(*, lock_secs, limit): RequestQueueHeadWithLocks
  • Parameters

    • lock_secs: intkeyword-only
    • limit: int | None = Nonekeyword-only

    Returns RequestQueueHeadWithLocks

list_head

  • async list_head(*, limit): RequestQueueHead
  • Parameters

    • limit: int | None = Nonekeyword-only

    Returns RequestQueueHead

prolong_request_lock

  • async prolong_request_lock(request_id, *, forefront, lock_secs): ProlongRequestLockResponse
  • Parameters

    • request_id: str
    • forefront: bool = Falsekeyword-only
    • lock_secs: intkeyword-only

    Returns ProlongRequestLockResponse

update

  • async update(*, name): RequestQueueMetadata
  • Parameters

    • name: str | None = Nonekeyword-only

    Returns RequestQueueMetadata

update_request

  • async update_request(request, *, forefront): ProcessedRequest
  • Parameters

    • request: Request
    • forefront: bool = Falsekeyword-only

    Returns ProcessedRequest

update_timestamps

  • async update_timestamps(*, has_been_modified): None
  • Update the timestamps of the request queue.


    Parameters

    • has_been_modified: boolkeyword-only

    Returns None

Properties

resource_directory

resource_directory: str

Get the resource directory for the client.

resource_info

resource_info: RequestQueueMetadata

Get the resource info for the request queue client.