StorageClient

Defines an abstract base for storage clients.

It offers interfaces to get subclients for interacting with storage resources like datasets, key-value stores, and request queues.

Hierarchy

• StorageClient
  • MemoryStorageClient

Index

Methods

• dataset
• datasets
• get_rate_limit_errors
• key_value_store
• key_value_stores
• purge_on_start
• request_queue
• request_queues

Methods

dataset

• dataset(id): DatasetClient

• Gets a subclient for a specific dataset by its ID.

  ---

  Parameters

  • id: str

  Returns DatasetClient

datasets

• datasets(): DatasetCollectionClient

• Gets a subclient for dataset collection operations.

  ---

  Returns DatasetCollectionClient

get_rate_limit_errors

• get_rate_limit_errors(): dict[int, int]

• Returns statistics about rate limit errors encountered by the HTTP client in storage client.

  ---

  Returns dict[int, int]

key_value_store

• key_value_store(id): KeyValueStoreClient

• Gets a subclient for a specific key-value store by its ID.

  ---

  Parameters

  • id: str

  Returns KeyValueStoreClient

key_value_stores

• key_value_stores(): KeyValueStoreCollectionClient

• Gets a subclient for key-value store collection operations.

  ---

  Returns KeyValueStoreCollectionClient

purge_on_start

• async purge_on_start(): None

• Performs a purge of the default storages.

  This method ensures that the purge is executed only once during the lifetime of the instance. It is primarily used to clean up residual data from previous runs to maintain a clean state. If the storage client does not support purging, leave it empty.

  ---

  Returns None

request_queue

• request_queue(id): RequestQueueClient

• Gets a subclient for a specific request queue by its ID.

  ---

  Parameters

  • id: str

  Returns RequestQueueClient

request_queues

• request_queues(): RequestQueueCollectionClient

• Gets a subclient for request queue collection operations.

  ---

  Returns RequestQueueCollectionClient