DatasetClient

An abstract class for dataset storage clients.

Dataset clients provide an interface for accessing and manipulating dataset storage. They handle operations like adding and getting dataset items across different storage backends.

Storage clients are specific to the type of storage they manage (Dataset, KeyValueStore, RequestQueue), and can operate with various storage systems including memory, file system, databases, and cloud storage solutions.

This abstract class defines the interface that all specific dataset clients must implement.

Hierarchy

• DatasetClient
  • MemoryDatasetClient
  • FileSystemDatasetClient

Index

Methods

• drop
• get_data
• get_metadata
• iterate_items
• purge
• push_data

Methods

drop

• async drop(): None

• Overrides DatasetClient.drop

  Drop the whole dataset and remove all its items.

  The backend method for the Dataset.drop call.

  ---

  Returns None

get_data

• async get_data(*, offset, limit, clean, desc, fields, omit, unwind, skip_empty, skip_hidden, flatten, view): DatasetItemsListPage

• Overrides DatasetClient.get_data

  Get data from the dataset with various filtering options.

  The backend method for the Dataset.get_data call.

  ---

  Parameters

  • optionalkeyword-onlyoffset: int = 0

  • optionalkeyword-onlylimit: int | None = 999_999_999_999

  • optionalkeyword-onlyclean: bool = False

  • optionalkeyword-onlydesc: bool = False

  • optionalkeyword-onlyfields: list[str] | None = None

  • optionalkeyword-onlyomit: list[str] | None = None

  • optionalkeyword-onlyunwind: str | None = None

  • optionalkeyword-onlyskip_empty: bool = False

  • optionalkeyword-onlyskip_hidden: bool = False

  • optionalkeyword-onlyflatten: list[str] | None = None

  • optionalkeyword-onlyview: str | None = None

  Returns DatasetItemsListPage

get_metadata

• async get_metadata(): DatasetMetadata

• Overrides DatasetClient.get_metadata

  Get the metadata of the dataset.

  ---

  Returns DatasetMetadata

iterate_items

• async iterate_items(*, offset, limit, clean, desc, fields, omit, unwind, skip_empty, skip_hidden): AsyncIterator[dict[str, Any]]

• Overrides DatasetClient.iterate_items

  Iterate over the dataset items with filtering options.

  The backend method for the Dataset.iterate_items call.

  ---

  Parameters

  • optionalkeyword-onlyoffset: int = 0

  • optionalkeyword-onlylimit: int | None = None

  • optionalkeyword-onlyclean: bool = False

  • optionalkeyword-onlydesc: bool = False

  • optionalkeyword-onlyfields: list[str] | None = None

  • optionalkeyword-onlyomit: list[str] | None = None

  • optionalkeyword-onlyunwind: str | None = None

  • optionalkeyword-onlyskip_empty: bool = False

  • optionalkeyword-onlyskip_hidden: bool = False

  Returns AsyncIterator[dict[str, Any]]

purge

• async purge(): None

• Overrides DatasetClient.purge

  Purge all items from the dataset.

  The backend method for the Dataset.purge call.

  ---

  Returns None

push_data

• async push_data(data): None

• Overrides DatasetClient.push_data

  Push data to the dataset.

  The backend method for the Dataset.push_data call.

  ---

  Parameters

  • data: list[Any] | dict[str, Any]

  Returns None