Helper ClassesAddRequestsKwargsOn this pageAddRequestsKwargs Keyword arguments for crawler's add_requests method.HierarchyAddRequestsKwargsAddRequestsFunctionCallIndex Propertiesbase_urlexcludeincludelimitstrategyProperties base_urlbase_url: NotRequired[str]excludeexclude: NotRequired[list[re.Pattern | Glob]]includeinclude: NotRequired[list[re.Pattern | Glob]]limitlimit: NotRequired[int]strategystrategy: NotRequired[EnqueueStrategy]
Keyword arguments for crawler's
add_requests
method.