_ProxyTierTracker

Tracks the state of currently used proxy tiers and their error frequency for individual crawled domains.

Index

Methods

• __init__
• add_error
• get_tier_urls
• predict_tier

Properties

• all_urls

Methods

__init__

• __init__(*, tiered_proxy_urls): None

• Parameters

  • optionalkeyword-onlytiered_proxy_urls: list[list[URL | None]]

  Returns None

add_error

• add_error(*, domain, tier): None

• Parameters

  • optionalkeyword-onlydomain: str

  • optionalkeyword-onlytier: int

  Returns None

get_tier_urls

• get_tier_urls(*, tier_number): Sequence[URL | None]

• Parameters

  • optionalkeyword-onlytier_number: int

  Returns Sequence[URL | None]

predict_tier

• predict_tier(*, domain): int

• Parameters

  • optionalkeyword-onlydomain: str

  Returns int

Properties

all_urls

all_urls: Sequence[URL | None]