RequestQueueClient

crawlee.memory_storage_client._request_queue_client.RequestQueueClient

Subclient for manipulating a single request queue.

Index

Constructors

• __init__

Methods

• add_request
• batch_add_requests
• batch_delete_requests
• delete
• delete_request
• delete_request_lock
• get
• get_request
• list_and_lock_head
• list_head
• prolong_request_lock
• update
• update_request
• update_timestamps

Properties

• resource_directory
• resource_info

Constructors

__init__

• __init__(*, memory_storage_client, id, name, created_at, accessed_at, modified_at, handled_request_count, pending_request_count): None

• Parameters

  • memory_storage_client: MemoryStorageClientkeyword-only

  • id: str | None = Nonekeyword-only

  • name: str | None = Nonekeyword-only

  • created_at: datetime | None = Nonekeyword-only

  • accessed_at: datetime | None = Nonekeyword-only

  • modified_at: datetime | None = Nonekeyword-only

  • handled_request_count: int = 0keyword-only

  • pending_request_count: int = 0keyword-only

  Returns None

Methods

add_request

• async add_request(request, *, forefront): ProcessedRequest

• Parameters

  • request: Request

  • forefront: bool = Falsekeyword-only

  Returns ProcessedRequest

batch_add_requests

• async batch_add_requests(requests, *, forefront): BatchRequestsOperationResponse

• Parameters

  • requests: Sequence[Request]

  • forefront: bool = Falsekeyword-only

  Returns BatchRequestsOperationResponse

batch_delete_requests

• async batch_delete_requests(requests): BatchRequestsOperationResponse

• Parameters

  • requests: list[Request]

  Returns BatchRequestsOperationResponse

delete

• async delete(): None

• Returns None

delete_request

• async delete_request(request_id): None

• Parameters

  • request_id: str

  Returns None

delete_request_lock

• async delete_request_lock(request_id, *, forefront): None

• Parameters

  • request_id: str

  • forefront: bool = Falsekeyword-only

  Returns None

get

• async get(): RequestQueueMetadata | None

• Returns RequestQueueMetadata | None

get_request

• async get_request(request_id): Request | None

• Parameters

  • request_id: str

  Returns Request | None

list_and_lock_head

• async list_and_lock_head(*, lock_secs, limit): RequestQueueHeadWithLocks

• Parameters

  • lock_secs: intkeyword-only

  • limit: int | None = Nonekeyword-only

  Returns RequestQueueHeadWithLocks

list_head

• async list_head(*, limit): RequestQueueHead

• Parameters

  • limit: int | None = Nonekeyword-only

  Returns RequestQueueHead

prolong_request_lock

• async prolong_request_lock(request_id, *, forefront, lock_secs): ProlongRequestLockResponse

• Parameters

  • request_id: str

  • forefront: bool = Falsekeyword-only

  • lock_secs: intkeyword-only

  Returns ProlongRequestLockResponse

update

• async update(*, name): RequestQueueMetadata

• Parameters

  • name: str | None = Nonekeyword-only

  Returns RequestQueueMetadata

update_request

• async update_request(request, *, forefront): ProcessedRequest

• Parameters

  • request: Request

  • forefront: bool = Falsekeyword-only

  Returns ProcessedRequest

update_timestamps

• async update_timestamps(*, has_been_modified): None

• Update the timestamps of the request queue.

  ---

  Parameters

  • has_been_modified: boolkeyword-only

  Returns None

Properties

resource_directory

resource_directory: str

Get the resource directory for the client.

resource_info

resource_info: RequestQueueMetadata

Get the resource info for the request queue client.