RenderingTypePredictor

Stores rendering type information for previously crawled URLs and predicts the rendering type for URLs that have yet to be crawled and recommends when rendering type detection should be performed.

Implements

• IRenderingTypePredictor

Index

Constructors

• constructor

Methods

• [asyncDispose]
• initialize
• predict
• storeResult
• teardown

Constructors

constructor

• new RenderingTypePredictor(__namedParameters): RenderingTypePredictor

• Parameters

  • __namedParameters: RenderingTypePredictorOptions

  Returns RenderingTypePredictor

Methods

[asyncDispose]

• [asyncDispose](): Promise<void>

• Returns Promise<void>

initialize

• initialize(): Promise<void>

• Initialize the predictor by restoring persisted state.

  ---

  Returns Promise<void>

publicpredict

• predict(__namedParameters): { detectionProbabilityRecommendation: number; renderingType: RenderingType }

• Implementation of IRenderingTypePredictor.predict

  Predict the rendering type for a given URL and request label.

  ---

  Parameters

  • __namedParameters: CrawleeRequest<Dictionary>

  Returns { detectionProbabilityRecommendation: number; renderingType: RenderingType }

  • detectionProbabilityRecommendation: number

  • renderingType: RenderingType

publicstoreResult

• storeResult(requests, renderingType): void

• Implementation of IRenderingTypePredictor.storeResult

  Store the rendering type for a given URL and request label. This updates the underlying prediction model, which may be costly.

  ---

  Parameters

  • requests: CrawleeRequest<Dictionary> | CrawleeRequest<Dictionary>[]

  • renderingType: RenderingType

  Returns void

teardown

• teardown(): Promise<void>

• Stop persisting the model, writing it out one last time. initialize() reopens the persistence window.

  ---

  Returns Promise<void>