IStatistics <StateExtension>
Implemented by
Index
Properties
readonlyerrorTracker
Tracker for errors on the final retry of a request.
readonlyerrorTrackerRetry
Tracker for errors on retries prior to the final one.
readonlyrequestRetryHistogram
Retries histogram - index i holds the number of requests that finished after i retries.
readonlystate
The live statistics state the crawler reads for status messages and the final summary.
Methods
calculate
Computes the derived aggregates (averages, per-minute rates, totals) from the current state.
Returns CalculatedStatistics
discardJob
Drops a started request without counting it as finished or failed (e.g. skipped by robots.txt).
Parameters
id: string | number
Returns void
failJob
Marks a started request as failed, updating the failed counters and durations.
Parameters
id: string | number
retryCount: number
Returns void
finishJob
Marks a started request as finished, updating the finished counters and durations.
Parameters
id: string | number
retryCount: number
Returns void
optionalpersistState
Persists the current state to the key-value store. Optional - the crawler calls it on migration, but a backend with no persistence of its own can omit it.
Returns Promise<void>
registerStatusCode
Increments the counter for the given HTTP status code.
Parameters
code: number
Returns void
startCapturing
Begins a capture window: loads any persisted state, subscribes to persistence events, starts periodic logging.
Returns Promise<void>
startJob
Marks a request as started, so its duration can be measured on finish/fail.
Parameters
id: string | number
Returns void
stopCapturing
Ends the capture window: stops logging, unsubscribes, and persists the final state.
Returns Promise<void>
The statistics surface a crawler depends on: recording per-request outcomes, tracking errors, and driving the capture lifecycle for a run. Injected via the crawler's
statisticsoption, so a custom implementation can be plugged in without subclassing the crawler.StateExtensiondescribes the custom fields tracked alongside the built-in StatisticState ones - see StatisticsOptions.stateExtension.The owned-only mutators the crawler uses to own a default it built -
reset()/resetStore()- are deliberately absent: an injected instance is borrowed, and the crawler never wipes it. Those live on the concrete Statistics only.