XML Sitemap

An XML sitemap is a file that lists all the important URLs on a website, helping search engines discover and index pages.

In web scraping, sitemaps are a fast way to enumerate all URLs on a target site without crawling every link — feed the sitemap URL to a scraper to instantly get the full URL list.