Run a sitemap locally

Before running a local scrape

  • Open the sitemap in the Advanced Sitemap Builder.
  • Preview the relevant selectors and confirm that they return the expected values.
  • For an initial test, use a small set of start URLs where practical.
  • Confirm that the target website loads normally in the browser.

Run the sitemap

  1. Open the sitemap in the Advanced Sitemap Builder.
  2. Select 'Scrape' in the sitemap dropdown panel.
  3. Configure the request interval.
  4. Configure the page load delay if the website needs additional time to render content.
  5. Start the scraping job.

Web Scraper opens a scraping window and uses it to load pages and execute the sitemap. Keep the scraping window open until the job finishes. Closing the browser, allowing the computer to sleep, losing the network connection or otherwise interrupting the browser can stop the job.

After the job finishes

  1. Open the scraped-data view.
  2. Inspect the resulting records, including data from later pages when the sitemap uses pagination or navigation.
  3. Export the data in the required local format.

When to use Web Scraper Cloud

Use Web Scraper Cloud when the job needs to run unattended or on a schedule, or when you need to process large volumes of data or use Cloud features such as proxies, monitoring, retries, API access, webhooks or automated exports.

Related