Sitemap

Index

Methods

• __init__
• from_xml_string
• load
• parse
• try_common_names

Properties

• urls

Methods

__init__

• __init__(urls): None

• Parameters

  • urls: list[str]

  Returns None

from_xml_string

• async from_xml_string(content, *, sitemap_url, parse_sitemap_options): Sitemap

• Parse a sitemap from a raw XML string.

  ---

  Parameters

  • content: str

    The sitemap XML content.

  • optionalkeyword-onlysitemap_url: str | None = None

    URL the content was retrieved from. Providing it enables the same URL filtering as for URL-loaded sitemaps: entries are kept only if they match the enqueue strategy (same-hostname by default) relative to this URL. Without it, no filtering is applied.

  • optionalkeyword-onlyparse_sitemap_options: ParseSitemapOptions | None = None

    Options for parsing, see ParseSitemapOptions.

  Returns Sitemap

load

• async load(urls, http_client, proxy_info, parse_sitemap_options): Sitemap

• Parameters

  • urls: str | list[str]

  • http_client: HttpClient

  • optionalproxy_info: ProxyInfo | None = None

  • optionalparse_sitemap_options: ParseSitemapOptions | None = None

  Returns Sitemap

parse

• async parse(sources, http_client, proxy_info, parse_sitemap_options): Sitemap

• Parameters

  • sources: list[SitemapSource]

  • optionalhttp_client: HttpClient | None = None

  • optionalproxy_info: ProxyInfo | None = None

  • optionalparse_sitemap_options: ParseSitemapOptions | None = None

  Returns Sitemap

try_common_names

• async try_common_names(url, http_client, proxy_info): Sitemap

• Parameters

  • url: str

  • http_client: HttpClient

  • optionalproxy_info: ProxyInfo | None = None

  Returns Sitemap

Properties

urls

urls: list[str]