Scheduler is a feature for Web Scraper Cloud which enables the user to automatize scraping jobs for a specific
time. It can be set by navigating to sitemap
Details page and clicking on
Note! If an ongoing scraping job is in motion or it runs for longer than the scheduled interval, the new scheduled scraping job will start only when the previous one has finished.
The configuration options for scheduled sitemap scraping jobs are:
These configuration options will be applied for all scheduled scraping jobs, for the specific sitemap.
There are 3 types of scheduler configuration options available:
Select the weekdays and the time for which the scraping jobs should be started.
Interval can be set either by hours or minutes. If the interval is set by hours, scheduler will start scraping jobs on the round hours.
If a very customized and specific scheduler is needed, for which the previous scheduler types are not fit for, set up a custom Cron expression.