DatasetClient
crawlee.memory_storage_client._dataset_client.DatasetClient
Index
Constructors
__init__
Parameters
memory_storage_client: MemoryStorageClientkeyword-only
id: str | None = Nonekeyword-only
name: str | None = Nonekeyword-only
created_at: datetime | None = Nonekeyword-only
accessed_at: datetime | None = Nonekeyword-only
modified_at: datetime | None = Nonekeyword-only
item_count: int = 0keyword-only
Returns None
Methods
delete
Returns None
get
Returns DatasetMetadata | None
get_items_as_bytes
Parameters
item_format: str = 'json'keyword-only
offset: int | None = Nonekeyword-only
limit: int | None = Nonekeyword-only
desc: bool = Falsekeyword-only
clean: bool = Falsekeyword-only
bom: bool = Falsekeyword-only
delimiter: str | None = Nonekeyword-only
fields: list[str] | None = Nonekeyword-only
omit: list[str] | None = Nonekeyword-only
unwind: str | None = Nonekeyword-only
skip_empty: bool = Falsekeyword-only
skip_header_row: bool = Falsekeyword-only
skip_hidden: bool = Falsekeyword-only
xml_root: str | None = Nonekeyword-only
xml_row: str | None = Nonekeyword-only
flatten: list[str] | None = Nonekeyword-only
Returns bytes
get_start_and_end_indexes
Calculate the start and end indexes for listing items.
Parameters
offset: int
limit: int | None = None
Returns tuple[int, int]
iterate_items
Parameters
offset: int = 0keyword-only
limit: int | None = Nonekeyword-only
clean: bool = Falsekeyword-only
desc: bool = Falsekeyword-only
fields: list[str] | None = Nonekeyword-only
omit: list[str] | None = Nonekeyword-only
unwind: str | None = Nonekeyword-only
skip_empty: bool = Falsekeyword-only
skip_hidden: bool = Falsekeyword-only
Returns AsyncIterator[dict]
list_items
Parameters
offset: int | None = 0keyword-only
limit: int | None = _LIST_ITEMS_LIMITkeyword-only
clean: bool = Falsekeyword-only
desc: bool = Falsekeyword-only
fields: list[str] | None = Nonekeyword-only
omit: list[str] | None = Nonekeyword-only
unwind: str | None = Nonekeyword-only
skip_empty: bool = Falsekeyword-only
skip_hidden: bool = Falsekeyword-only
flatten: list[str] | None = Nonekeyword-only
view: str | None = Nonekeyword-only
Returns DatasetItemsListPage
push_items
Parameters
items: JsonSerializable
Returns None
stream_items
Parameters
item_format: str = 'json'keyword-only
offset: int | None = Nonekeyword-only
limit: int | None = Nonekeyword-only
desc: bool = Falsekeyword-only
clean: bool = Falsekeyword-only
bom: bool = Falsekeyword-only
delimiter: str | None = Nonekeyword-only
fields: list[str] | None = Nonekeyword-only
omit: list[str] | None = Nonekeyword-only
unwind: str | None = Nonekeyword-only
skip_empty: bool = Falsekeyword-only
skip_header_row: bool = Falsekeyword-only
skip_hidden: bool = Falsekeyword-only
xml_root: str | None = Nonekeyword-only
xml_row: str | None = Nonekeyword-only
Returns AsyncContextManager[Response | None]
update
Parameters
name: str | None = Nonekeyword-only
Returns DatasetMetadata
update_timestamps
Update the timestamps of the dataset.
Parameters
has_been_modified: boolkeyword-only
Returns None
Properties
resource_directory
Get the resource directory for the client.
resource_info
Get the resource info for the dataset client.
Subclient for manipulating a single dataset.