Skip to main content
Version: 3.1

MemoryStorage

Represents a storage capable of working with datasets, KV stores and request queues.

Implements

  • StorageClient

Index

Constructors

Properties

Methods

Constructors

constructor

  • new MemoryStorage(options?: MemoryStorageOptions): MemoryStorage

  • Parameters

    • options: MemoryStorageOptions = {}

    Returns MemoryStorage

Properties

readonlydatasetClientsHandled

datasetClientsHandled: DatasetClient<Dictionary<any>>[] = []

readonlydatasetsDirectory

datasetsDirectory: string

readonlykeyValueStoresDirectory

keyValueStoresDirectory: string

readonlykeyValueStoresHandled

keyValueStoresHandled: KeyValueStoreClient[] = []

readonlylocalDataDirectory

localDataDirectory: string

readonlypersistStorage

persistStorage: boolean

readonlyrequestQueuesDirectory

requestQueuesDirectory: string

readonlyrequestQueuesHandled

requestQueuesHandled: RequestQueueClient[] = []

readonlywriteMetadata

writeMetadata: boolean

Methods

dataset

  • dataset<Data>(id: string): DatasetClient<Data>

  • Type parameters

    • Data: Dictionary<any> = Dictionary<any>

    Parameters

    • id: string

    Returns DatasetClient<Data>

datasets

  • datasets(): DatasetCollectionClient

  • Returns DatasetCollectionClient

keyValueStore

  • keyValueStore(id: string): KeyValueStoreClient

  • Parameters

    • id: string

    Returns KeyValueStoreClient

keyValueStores

  • keyValueStores(): KeyValueStoreCollectionClient

  • Returns KeyValueStoreCollectionClient

purge

  • purge(): Promise<void>

  • Cleans up the default storage directories before the run starts:

    • local directory containing the default dataset;
    • all records from the default key-value store in the local directory, except for the "INPUT" key;
    • local directory containing the default request queue.

    Returns Promise<void>

requestQueue

  • requestQueue(id: string, options?: RequestQueueOptions): RequestQueueClient

  • Parameters

    • id: string
    • options: RequestQueueOptions = {}

    Returns RequestQueueClient

requestQueues

  • requestQueues(): RequestQueueCollectionClient

  • Returns RequestQueueCollectionClient

teardown

  • teardown(): Promise<void>

  • This method should be called at the end of the process, to ensure all data is saved.

    Returns Promise<void>