Skip to main content

ProxyTierTracker

crawlee.proxy_configuration.ProxyTierTracker

Tracks the state of currently used proxy tiers and their error frequency for individual crawled domains.

Index

Errors

Constructors

Methods

Properties

Errors

add_error

  • add_error(domain, tier): None
  • Parameters

    • domain: str
    • tier: int

    Returns None

Constructors

__init__

  • __init__(tiered_proxy_urls): None
  • Parameters

    • tiered_proxy_urls: list[list[URL]]

    Returns None

Methods

get_tier_urls

  • get_tier_urls(tier_number): Sequence[URL]
  • Parameters

    • tier_number: int

    Returns Sequence[URL]

predict_tier

  • predict_tier(domain): int
  • Parameters

    • domain: str

    Returns int

Properties

all_urls

all_urls: Sequence[URL]