BaseDatasetCollectionClient

crawlee.base_storage_client._base_dataset_collection_client.BaseDatasetCollectionClient

Abstract base class for dataset collection clients.

This collection client handles operations that involve multiple instances of a given resource type.

Index

Methods

• get_or_create
• list

Methods

get_or_create

• async get_or_create(*, id, name, schema): DatasetMetadata

• Retrieve an existing dataset by its name or ID, or create a new one if it does not exist.

  ---

  Parameters

  • id: str | None = Nonekeyword-only

  • name: str | None = Nonekeyword-only

  • schema: dict | None = Nonekeyword-only

  Returns DatasetMetadata

list

• async list(*, unnamed, limit, offset, desc): DatasetListPage

• List the available datasets.

  ---

  Parameters

  • unnamed: bool = Falsekeyword-only

  • limit: int | None = Nonekeyword-only

  • offset: int | None = Nonekeyword-only

  • desc: bool = Falsekeyword-only

  Returns DatasetListPage