MemoryStorageClient
Hierarchy
- StorageClient
- MemoryStorageClient
Index
Methods
Properties
Methods
__init__
Initialize a new instance.
In most cases, you should use the
from_configconstructor to create a new instance based on the provided configuration.Parameters
keyword-onlywrite_metadata: bool
Whether to write metadata to the storage.
keyword-onlypersist_storage: bool
Whether to persist the storage.
keyword-onlystorage_dir: str
Path to the storage directory.
keyword-onlydefault_request_queue_id: str
The default request queue ID.
keyword-onlydefault_key_value_store_id: str
The default key-value store ID.
keyword-onlydefault_dataset_id: str
The default dataset ID.
Returns None
add_resource_client_to_cache
Add a new resource client to the internal cache.
Parameters
resource_client: ResourceClient
Returns None
dataset
Get a subclient for a specific dataset by its ID.
Parameters
id: str
Returns DatasetClient
datasets
Get a subclient for dataset collection operations.
Returns DatasetCollectionClient
from_config
Initialize a new instance based on the provided
Configuration.Parameters
optionalconfig: Configuration | None = None
The
Configurationinstance. Uses the global (default) one if not provided.
Returns MemoryStorageClient
get_cached_resource_client
Try to return a resource client from the internal cache.
Parameters
resource_client_class: type[TResourceClient]
id: str | None
name: str | None
Returns TResourceClient | None
get_rate_limit_errors
Return statistics about rate limit errors encountered by the HTTP client in storage client.
Returns dict[int, int]
key_value_store
Get a subclient for a specific key-value store by its ID.
Parameters
id: str
Returns KeyValueStoreClient
key_value_stores
Get a subclient for key-value store collection operations.
Returns KeyValueStoreCollectionClient
purge_on_start
Perform a purge of the default storages.
This method ensures that the purge is executed only once during the lifetime of the instance. It is primarily used to clean up residual data from previous runs to maintain a clean state. If the storage client does not support purging, leave it empty.
Returns None
request_queue
Get a subclient for a specific request queue by its ID.
Parameters
id: str
Returns RequestQueueClient
request_queues
Get a subclient for request queue collection operations.
Returns RequestQueueCollectionClient
Properties
datasets_directory
Path to the directory containing datasets.
key_value_stores_directory
Path to the directory containing key-value stores.
persist_storage
Whether to persist the storage.
request_queues_directory
Path to the directory containing request queues.
storage_dir
Path to the storage directory.
write_metadata
Whether to write metadata to the storage.
Represents an in-memory storage client for managing datasets, key-value stores, and request queues.
It emulates in-memory storage similar to the Apify platform, supporting both in-memory and local file system-based persistence.
The behavior of the storage, such as data persistence and metadata writing, can be customized via initialization parameters or environment variables.