Skip to main content

GetDataKwargs

crawlee.storages.dataset.GetDataKwargs

Keyword arguments for dataset's get_data method.

Args: offset: Skips the specified number of items at the start. limit: The maximum number of items to retrieve. Unlimited if None. clean: Returns only non-empty items and excludes hidden fields. Shortcut for skip_hidden and skip_empty. desc: Set True to sort results in descending order. fields: Fields to include in each item. Sorts fields as specified if provided. omit: Fields to exclude from each item. unwind: Unwinds items by a specified array field, turning each element into a separate item. skip_empty: Excludes empty items from the results if True. skip_hidden: Excludes fields starting with '#' if True. flatten: Fields to be flattened in returned items. view: Specifies the dataset view to be used.

Index

Properties

clean

clean: NotRequired[bool]

desc

desc: NotRequired[bool]

fields

fields: NotRequired[list[str]]

flatten

flatten: NotRequired[list[str]]

limit

limit: NotRequired[int]

offset

offset: NotRequired[int]

omit

omit: NotRequired[list[str]]

skip_empty

skip_empty: NotRequired[bool]

skip_hidden

skip_hidden: NotRequired[bool]

unwind

unwind: NotRequired[str]

view

view: NotRequired[str]