DefaultRenderingTypePredictor

Stores rendering type for previously crawled URLs and predicts the rendering type for unvisited urls.

RenderingTypePredictor implementation based on logistic regression: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

Hierarchy

• RenderingTypePredictor
  • DefaultRenderingTypePredictor

Index

Methods

• __aenter__
• __aexit__
• __init__
• clear
• initialize
• predict
• store_result

Methods

__aenter__

• async __aenter__(): RenderingTypePredictor

• Inherited from RenderingTypePredictor.__aenter__

  Initialize the predictor upon entering the context manager.

  ---

  Returns RenderingTypePredictor

__aexit__

• async __aexit__(exc_type, exc_value, exc_traceback): None

• Inherited from RenderingTypePredictor.__aexit__

  Clear the predictor upon exiting the context manager.

  ---

  Parameters

  • exc_type: type[BaseException] | None

  • exc_value: BaseException | None

  • exc_traceback: TracebackType | None

  Returns None

__init__

• __init__(detection_ratio, *, persistence_enabled, persist_state_key): None

• Overrides RenderingTypePredictor.__init__

  Initialize a new instance.

  ---

  Parameters

  • optionaldetection_ratio: float = 0.1

    A number between 0 and 1 that determines the desired ratio of rendering type detections.

  • optionalkeyword-onlypersistence_enabled: bool = False

    Whether to enable persistence of the trained model parameters for reuse.

  • optionalkeyword-onlypersist_state_key: str = 'rendering-type-predictor-state'

    Key in the key-value storage where the trained model parameters will be saved. If None, defaults to 'rendering-type-predictor-state'.

  Returns None

clear

• async clear(): None

• Overrides RenderingTypePredictor.clear

  Clear the predictor state.

  ---

  Returns None

initialize

• async initialize(): None

• Overrides RenderingTypePredictor.initialize

  Get current state of the predictor.

  ---

  Returns None

predict

• predict(request): RenderingTypePrediction

• Overrides RenderingTypePredictor.predict

  Get RenderingTypePrediction based on the input request.

  ---

  Parameters

  • request: Request

    Request instance for which the prediction is made.

  Returns RenderingTypePrediction

store_result

• store_result(request, rendering_type): None

• Overrides RenderingTypePredictor.store_result

  Store prediction results and retrain the model.

  ---

  Parameters

  • request: Request

    Used Request instance.

  • rendering_type: RenderingType

    Known suitable RenderingType for the used Request instance.

  Returns None