Blog

How to scrape G2 company listings using Web Scraper
How to Scrape G2 Company data (Step-by-Step Guide)

March 06, 2026

Company directories on G2 list software vendors across a wide range of product categories and provide structured information about companies, ratings, and market presence. This data is often used to analyse software ecosystems, identify competitors, and build datasets of companies operating in specific technology segments. In this guide, you will learn how to scrape G2 company listings using Web Scraper and extract structured company data directly from directory pages without writing code. The extracted data can be exported in CSV, Excel, or JSON for further analysis, storage, or integration into other systems.

Continue Reading

How to scrape Rightmove property listing data using Web Scraper
How to Scrape Rightmove Property Listings data (Step-by-Step Guide)

March 06, 2026

Rightmove is one of the largest online real estate platforms in the United Kingdom, used to explore homes for sale, rental properties, and housing market listings across different regions. Property listings on Rightmove contain structured information such as listing prices, property details, locations, images, and real estate agent information. In this guide, you will learn how to scrape Rightmove property listings using Web Scraper and extract structured real estate data directly from search result pages and property detail pages without writing code. The extracted data can be exported in CSV, Excel, or JSON formats for further analysis, real estate market research, price monitoring, or integration into other systems.

Continue Reading

Robot solving verify you are human captcha
Google patches 100% precise Cloudflare Turnstile bot check

October 24, 2025

Bypassing Cloudflare Turnstile captcha is not easy for DIY scrapers. Recently, Cloudflare has been upping its game by releasing innovative bot detection solutions that might feel undefeatable. Most bots are based on Chrome, and one of the best approaches to detect them is to check whether the browser is automated using Chrome Devtools Protocol (CDP). No regular Chrome user would have this protocol enabled, while 99% of bots will be using this to automate Chrome. Cloudflare came up with a new way to detect this by exploiting a bug in the Chrome browser, and Google is fixing it.

Continue Reading

Web Scraper Sitemap Wizard
Update: Sitemap Wizard, AI-Powered Selector Generation and more

August 08, 2025

Advanced Sitemap Builder, Reliable web scraping selectors, Multi-element data extraction using AI, AI-powered selector generation, AI web scraping automation, Sitemap Wizard for web scraping, Web Scraper AI update, Automated data extraction tools, Streamlined web scraper interface with AI, CSS selector improvements

We are excited to announce a new version of the Web Scraper extension packed with AI-powered improvements, usability upgrades, and some important changes to how selectors work. This release focuses on making your most common use cases simpler while utilising AI tools for advanced data extraction scenarios.

Continue Reading

Website State Setup release
Website State Setup: Expand your scrapers functionality

August 08, 2024

Authenticated page scraping, Login automation for web scraping, Web Scraper Website State Setup, Web Scraper change currency feature, Web Scraper input and click automation, Conditional data extraction Web Scraper, Automate conditional web scraping actions, Web Scraper change location feature, Website-specific data collection automation, Efficient web scraping workflow

In the world of web scraping and data collection, efficiency and precision are paramount. That's why we're excited about the new feature that is being added to Web Scraper functionality: Website State Setup. This feature you to execute conditional actions based on specific criteria, ensuring you collect the most relevant data.

Continue Reading

Web Scraper 1.79.3 release
Web Scraper 1.79.3 release

August 01, 2024

Page load detection fix, Web Scraper performance update, Web scraping tool enhancements, Web Scraper bug fixes, Input validation refactor, Web Scraper 1.79.3 release, Efficient web scraping features, Web Scraper sitemap sync, Optimized data export Web Scraper, Improved link extraction Web Scraper

New Update: Web Scraper 1.79.3

Web scraping enthusiasts and data aficionados, rejoice! The latest release of Web Scraper, version 1.79.3, brings several enhancements and fixes.

Continue Reading

Web Scraper 1.75.7 release
Web Scraper 1.75.7 release - post-click link access

February 19, 2024

Pagination type Link from script, Web Scraper update, Dynamic link scraping, Network traffic page navigation, Advanced link extraction, Window.open() and window.location scraping, Selector grouping feature, Misconfiguration modal for selectors, Web Scraper performance enhancements

Exciting Updates Alert!

We're thrilled to announce some powerful features and enhancements in our latest Web Scraper update!

Check out what's new:

Continue Reading

Web Scraper 1.72.5 release - UI updates and more

November 27, 2023

Web Scraper UI improvements, Web Scraper extension update, Enhanced Link Selector feature, HTML code on hover, Page load scraping fix, Scrape non-HTML links, Faster data extraction Chrome, Web Scraper user experience update, Refactored scraping extension, Selector deletion confirmation popup

We're thrilled to share the latest enhancements in our extension to make your experience even smoother. Check out the new features and improvements in our latest release:

Continue Reading

Web Scraper 1.29.66 release
Web Scraper 1.29.66 release - Updated Link Selector

July 07, 2023

Chrome MV3 compatibility update, Improved mouse click simulation, Web Scraper latest update, Web Scraper extension feature release, New Link Selector feature, Extract links from attributes, Sitemap deletion confirmation, Pop-up Link Selector removal, Scrape text-based links

Hey there! We've got some major news to share with you that will up your scraping game. We're thrilled to introduce the latest and greatest version of Web Scraper. Take your experience to a new level with the newly updated Link Selector and more.

Continue Reading

Web Scraper 1.29.60 release
Web Scraper 1.29.60 release

May 10, 2023

Selector execution order control, Web Scraper CSS selector optimization, Chrome Manifest V3 update, Web Scraper extension updates, Web Scraper feature release, PouchDB to local storage migration, Latest Web Scraper extension updates, Automated CSS selector restructuring, Web Scraper validation engine update, Improve web scraper speed

It's time for our latest feature release article, and we can't wait to show you what we've been working on!

Continue Reading

Web Scraper 0.6.5 release

September 09, 2022

Web Scraper shadow-root support, Web Scraper element scroll selector, Web Scraper release notes, Web Scraper 0.6.5 update, Web Scraper data export improvements, Web Scraper bug fixes, Firefox sitemap sync Web Scraper, Web Scraper UTF-8 text handling, Web Scraper performance optimizations, Web Scraper new features

For the last few months, we have been working towards the Web Scraper 0.6.5 release, and finally, it is here!

Continue Reading

A quick guide to CSS and jQuery selectors image
A Quick Guide to CSS and jQuery Selectors for Web Scraper

June 28, 2022

DOM element selection techniques, Isolating specific page elements for scraping, CSS selectors for web scraping, Web Scraper Chrome tool selector guide, :contains() and :has() jQuery selectors, Attribute-based CSS selectors, Advanced element targeting for scraping, Chaining CSS and jQuery selectors, nth-child and nth-of-type selectors, jQuery selectors for data extraction

Selectors, when it comes to CSS and jQuery, refer to code you can use to interact with specific elements in the DOM. They are particularly useful when you want to style or act upon certain elements of the document without altering other elements. Through a careful use of selectors, you can manipulate a page exactly as intended or isolate the specific data you’re looking to scrape.

Continue Reading

data analysis using amazon athena
Data Analysis using Amazon Athena

May 02, 2022

Average price calculation in Athena, Analyze web scraping data in AWS, Web scraping data analysis with Athena, AWS data querying and processing, Amazon Athena SQL queries, Web Scraper Cloud data insights, Sync scraping results to Amazon S3, Query JSON data with Athena, Create Athena table from S3 JSON, Web Scraper Cloud S3 integration

Amazon Athena lets you use SQL to interactively query any data you have within an Amazon S3 Bucket. With Web Scraper Cloud, you can automatically sync web scraping job results to an Amazon S3 Bucket where you can then use Athena to quickly query this data with little additional overhead. In this article, we’re going to guide you through syncing Web Scraper Cloud with an S3 bucket and using Athena to generate and perform calculations on table data. While we’re covering a specific use case, you can adapt this same process to any other data set to gather insights.

Continue Reading

Headless Scraping with Cheerio

March 21, 2022

Extract HTML elements with Cheerio, Parse web page content in Node.js, Headless scraper setup tutorial, Web scraping with Axios and Cheerio, JavaScript web scraping techniques, Axios HTTP requests for web scraping, Automated data extraction using Cheerio, Spoof browser headers for scraping, Cheerio Node.js web scraping, Scrape dynamic pages without a browser

Cheerio is a node package that allows you to easily parse and extract elements from markup. Unlike a browser, Cheerio doesn’t produce a visual rendering, load external resources, execute JavaScript code, or apply CSS. As a result, it’s much faster than other solutions when it comes to scraping web pages for valuable data.

In this article, we’re going to specifically cover how to set up a headless scraper with Cheerio and spoof browser headers to access a wider range of web pages.

Continue Reading

Scraping public data
Scraping public data. Is it legal?

March 03, 2022

Legal web scraping GDPR compliance, Lawful data processing for scraping public data, Compliance tips for automated data extraction, Facebook vs BrandTotal web scraping lawsuit, HiQ vs LinkedIn web scraping case analysis, Scraping publicly available data lawfully, Platform Terms and Conditions web scraping, Scraping emails and phone numbers legally, GDPR guidelines for data scraping, Data minimisation principle for web scraping

We’ve all been hearing about the ongoing cases of companies failing lawsuits against web scraping activities. Worth mentioning are the cases of LinkedIn vs. HiQ on public data scraping and the case of Facebook Inc. vs. BrandTotal Ltd. on scraping data from Facebook. What is more interesting is that in one case it was ruled in favour of the data scraper, but in the other - of the company whose platform is being scraped. Seeing such cases with different outcomes raises questions, especially due to the recent General Data Protection Regulation (GDPR), on how lawful web scraping of publicly available data actually is? 

Continue Reading