RequestHandlerResult

experimental

A partial implementation of RestrictedCrawlingContext that stores parameters of calls to context methods for later inspection.

Index

Constructors

• constructor

Properties

• addRequests
• enqueueLinks
• getKeyValueStore
• pushData
• useState

Accessors

• calls
• datasetItems
• enqueuedUrlLists
• enqueuedUrls
• keyValueStoreChanges

Constructors

constructor

• new RequestHandlerResult(config: Configuration, crawleeStateKey: string): RequestHandlerResult

• Parameters

  • config: Configuration

  • crawleeStateKey: string

  Returns RequestHandlerResult

Properties

addRequests

addRequests: (requestsLike: readonly (string | ReadonlyObjectDeep<Partial<RequestOptions<Dictionary>> & { regex?: RegExp; requestsFromUrl?: string }> | ReadonlyObjectDeep<Request<Dictionary>>)[], options?: ReadonlyObjectDeep<RequestQueueOperationOptions>) => Promise<void> = ...

Type declaration

• • (requestsLike: readonly (string | ReadonlyObjectDeep<Partial<RequestOptions<Dictionary>> & { regex?: RegExp; requestsFromUrl?: string }> | ReadonlyObjectDeep<Request<Dictionary>>)[], options?: ReadonlyObjectDeep<RequestQueueOperationOptions>): Promise<void>

  • Parameters

    • requestsLike: readonly (string | ReadonlyObjectDeep<Partial<RequestOptions<Dictionary>> & { regex?: RegExp; requestsFromUrl?: string }> | ReadonlyObjectDeep<Request<Dictionary>>)[]

    • optionaloptions: ReadonlyObjectDeep<RequestQueueOperationOptions>

    Returns Promise<void>

enqueueLinks

enqueueLinks: (options?: ReadonlyObjectDeep<Omit<EnqueueLinksOptions, requestQueue>>) => Promise<unknown> = ...

Type declaration

• • (options?: ReadonlyObjectDeep<Omit<EnqueueLinksOptions, requestQueue>>): Promise<unknown>

  • Parameters

    • optionaloptions: ReadonlyObjectDeep<Omit<EnqueueLinksOptions, requestQueue>>

    Returns Promise<unknown>

getKeyValueStore

getKeyValueStore: (idOrName?: string) => Promise<Pick<KeyValueStore, id | name | getValue | getAutoSavedValue | setValue>> = ...

Type declaration

• • (idOrName?: string): Promise<Pick<KeyValueStore, id | name | getValue | getAutoSavedValue | setValue>>

  • Parameters

    • optionalidOrName: string

    Returns Promise<Pick<KeyValueStore, id | name | getValue | getAutoSavedValue | setValue>>

pushData

pushData: (data?: ReadonlyDeep<Dictionary | Dictionary[]>, datasetIdOrName?: string) => Promise<void> = ...

Type declaration

• • (data?: ReadonlyDeep<Dictionary | Dictionary[]>, datasetIdOrName?: string): Promise<void>

  • This function allows you to push data to a Dataset specified by name, or the one currently used by the crawler.

    Shortcut for crawler.pushData().

    ---

    Parameters

    • optionaldata: ReadonlyDeep<Dictionary | Dictionary[]>

      Data to be pushed to the default dataset.

    • optionaldatasetIdOrName: string

    Returns Promise<void>

useState

useState: <State>(defaultValue?: State) => Promise<State> = ...

Type declaration

• • <State>(defaultValue?: State): Promise<State>

  • Type parameters

    • State: Dictionary = Dictionary

    Parameters

    • optionaldefaultValue: State

    Returns Promise<State>

Accessors

calls

• get calls(): ReadonlyObjectDeep<{ addRequests: [requestsLike: readonly (string | ReadonlyObjectDeep<Partial<RequestOptions<Dictionary>> & { regex?: RegExp; requestsFromUrl?: string }> | ReadonlyObjectDeep<Request<Dictionary>>)[], options?: ReadonlyObjectDeep<RequestQueueOperationOptions>][]; enqueueLinks: [options?: ReadonlyObjectDeep<Omit<EnqueueLinksOptions, requestQueue>>][]; pushData: [data: ReadonlyDeep<Dictionary | Dictionary[]>, datasetIdOrName?: string][] }>

• A record of calls to RestrictedCrawlingContext.pushData, RestrictedCrawlingContext.addRequests, RestrictedCrawlingContext.enqueueLinks made by a request handler.

  ---

  Returns ReadonlyObjectDeep<{ addRequests: [requestsLike: readonly (string | ReadonlyObjectDeep<Partial<RequestOptions<Dictionary>> & { regex?: RegExp; requestsFromUrl?: string }> | ReadonlyObjectDeep<Request<Dictionary>>)[], options?: ReadonlyObjectDeep<RequestQueueOperationOptions>][]; enqueueLinks: [options?: ReadonlyObjectDeep<Omit<EnqueueLinksOptions, requestQueue>>][]; pushData: [data: ReadonlyDeep<Dictionary | Dictionary[]>, datasetIdOrName?: string][] }>

datasetItems

• get datasetItems(): readonly ReadonlyObjectDeep<{ datasetIdOrName?: string; item: Dictionary }>[]

• Items added to datasets by a request handler.

  ---

  Returns readonly ReadonlyObjectDeep<{ datasetIdOrName?: string; item: Dictionary }>[]

enqueuedUrlLists

• get enqueuedUrlLists(): readonly ReadonlyObjectDeep<{ label?: string; listUrl: string }>[]

• URL lists enqueued to the request queue by a request handler via RestrictedCrawlingContext.addRequests using the requestsFromUrl option.

  ---

  Returns readonly ReadonlyObjectDeep<{ label?: string; listUrl: string }>[]

enqueuedUrls

• get enqueuedUrls(): readonly ReadonlyObjectDeep<{ label?: string; url: string }>[]

• URLs enqueued to the request queue by a request handler, either via RestrictedCrawlingContext.addRequests or RestrictedCrawlingContext.enqueueLinks

  ---

  Returns readonly ReadonlyObjectDeep<{ label?: string; url: string }>[]

keyValueStoreChanges

• get keyValueStoreChanges(): ReadonlyObjectDeep<Record<string, Record<string, { changedValue: unknown; options?: RecordOptions }>>>

• A record of changes made to key-value stores by a request handler.

  ---

  Returns ReadonlyObjectDeep<Record<string, Record<string, { changedValue: unknown; options?: RecordOptions }>>>