MemoryStorageClient

Represents an in-memory storage client for managing datasets, key-value stores, and request queues.

It emulates in-memory storage similar to the Apify platform, supporting both in-memory and local file system-based persistence.

The behavior of the storage, such as data persistence and metadata writing, can be customized via initialization parameters or environment variables.

Hierarchy

• BaseStorageClient
  • MemoryStorageClient

Index

Constructors

• __init__

Methods

• add_resource_client_to_cache
• dataset
• datasets
• get_cached_resource_client
• key_value_store
• key_value_stores
• purge_on_start
• request_queue
• request_queues

Properties

• datasets_directory
• key_value_stores_directory
• persist_storage
• request_queues_directory
• storage_dir
• write_metadata

Constructors

__init__

• __init__(configuration): None

• Create a new instance.

  ---

  Parameters

  • configuration: Configuration | None = None

    Configuration object to use. If None, a default Configuration object will be created.

  Returns None

Methods

add_resource_client_to_cache

• add_resource_client_to_cache(resource_client): None

• Add a new resource client to the internal cache.

  ---

  Parameters

  • resource_client: ResourceClient

  Returns None

dataset

• dataset(id): DatasetClient

• Parameters

  • id: str

  Returns DatasetClient

datasets

• datasets(): DatasetCollectionClient

• Returns DatasetCollectionClient

get_cached_resource_client

• get_cached_resource_client(resource_client_class, id, name): TResourceClient | None

• Try to return a resource client from the internal cache.

  ---

  Parameters

  • resource_client_class: type[TResourceClient]

  • id: str | None

  • name: str | None

  Returns TResourceClient | None

key_value_store

• key_value_store(id): KeyValueStoreClient

• Parameters

  • id: str

  Returns KeyValueStoreClient

key_value_stores

• key_value_stores(): KeyValueStoreCollectionClient

• Returns KeyValueStoreCollectionClient

purge_on_start

• async purge_on_start(): None

• Returns None

request_queue

• request_queue(id): RequestQueueClient

• Parameters

  • id: str

  Returns RequestQueueClient

request_queues

• request_queues(): RequestQueueCollectionClient

• Returns RequestQueueCollectionClient

Properties

datasets_directory

datasets_directory: str

Path to the directory containing datasets.

key_value_stores_directory

key_value_stores_directory: str

Path to the directory containing key-value stores.

persist_storage

persist_storage: bool

Whether to persist the storage.

request_queues_directory

request_queues_directory: str

Path to the directory containing request queues.

storage_dir

storage_dir: str

Path to the storage directory.

write_metadata

write_metadata: bool

Whether to write metadata to the storage.