_DomainState

Tracks delay state for a single domain.

Index

Properties

• consecutive_429_count
• crawl_delay
• domain
• throttled_until

Properties

consecutive_429_count

consecutive_429_count: int

Number of consecutive 429 responses (for exponential backoff).

crawl_delay

crawl_delay: timedelta | None

Minimum interval between requests, used to push throttled_until on dispatch.

domain

domain: str

The domain being tracked.

throttled_until

throttled_until: datetime

Earliest time the next request to this domain is allowed.