Skip to main content
Version: 3.0

MemoryStorage

Represents a storage capable of working with datasets, KV stores and request queues.

Implements

  • StorageClient

Index

Constructors

constructor

  • new MemoryStorage(options?: MemoryStorageOptions): MemoryStorage
  • Parameters

    • options: MemoryStorageOptions = {}

    Returns MemoryStorage

Properties

readonlydatasetClientsHandled

datasetClientsHandled: DatasetClient<Dictionary<any>>[] = []

readonlydatasetsDirectory

datasetsDirectory: string

readonlykeyValueStoresDirectory

keyValueStoresDirectory: string

readonlykeyValueStoresHandled

keyValueStoresHandled: KeyValueStoreClient[] = []

readonlylocalDataDirectory

localDataDirectory: string

readonlyrequestQueuesDirectory

requestQueuesDirectory: string

readonlyrequestQueuesHandled

requestQueuesHandled: RequestQueueClient[] = []

readonlywriteMetadata

writeMetadata: boolean

Methods

dataset

  • dataset<Data>(id: string): DatasetClient<Data>
  • Type parameters

    • Data: Dictionary<any> = Dictionary<any>

    Parameters

    • id: string

    Returns DatasetClient<Data>

datasets

  • datasets(): DatasetCollectionClient
  • Returns DatasetCollectionClient

keyValueStore

  • keyValueStore(id: string): KeyValueStoreClient
  • Parameters

    • id: string

    Returns KeyValueStoreClient

keyValueStores

  • keyValueStores(): KeyValueStoreCollectionClient
  • Returns KeyValueStoreCollectionClient

purge

  • purge(): Promise<void>
  • Cleans up the default storage directories before the run starts:

    • local directory containing the default dataset;
    • all records from the default key-value store in the local directory, except for the "INPUT" key;
    • local directory containing the default request queue.

    Returns Promise<void>

requestQueue

  • requestQueue(id: string, options?: RequestQueueOptions): RequestQueueClient
  • Parameters

    • id: string
    • options: RequestQueueOptions = {}

    Returns RequestQueueClient

requestQueues

  • requestQueues(): RequestQueueCollectionClient
  • Returns RequestQueueCollectionClient

teardown

  • teardown(): Promise<void>
  • This method should be called at the end of the process, to ensure all data is saved.


    Returns Promise<void>