How Proxies Help You Scrape Reviews Data: Trustpilot, G2 & Google Business

Review data is the lifeblood of brand monitoring, competitive analysis, and reputation management. Whether you track Trustpilot ratings for a client's competitors, monitor G2 sentiment across SaaS categories, or aggregate Google Business reviews for multi-location analysis, you need reliable access to public review pages at scale. This guide explains how proxies help you scrape reviews data across multiple platforms — with runnable code examples for Trustpilot, G2, and Google Business.

Important caveat: This guide covers only publicly accessible review data — no login-walled content, no PII harvesting beyond what is necessary. You are responsible for complying with each platform's Terms of Service, robots.txt directives, and applicable laws including the U.S. Computer Fraud and Abuse Act (CFAA) and the EU General Data Protection Regulation (GDPR). When an official API is available and sufficient for your use case, prefer it over scraping.

Why Review Pages Are Aggressively Rate-Limited and Geo-Personalized

Review platforms invest heavily in anti-bot infrastructure. A single IP address hitting Trustpilot, G2, or Google Business pages at any meaningful volume will quickly encounter HTTP 429 responses, CAPTCHA challenges, or silent localization — where the server returns a different subset of reviews based on the requester's apparent geography. Understanding how proxies help scrape reviews starts with understanding why these barriers exist.

Three forces drive this behavior:

• Rate limiting: Platforms like G2 and Trustpilot throttle requests per IP to protect infrastructure and prevent automated data extraction. According to DataDome's bot protection documentation, modern anti-bot systems analyze request frequency, header consistency, and TLS fingerprints to identify automated traffic. Datacenter IPs may be blocked after as few as 50 requests.
• Geo-personalization: Google Business reviews surface differently depending on the hl (language) and gl (country) query parameters, as well as the IP's geolocation. A request from a German IP may see reviews prioritized in German, while a U.S. IP sees English reviews first. This makes a single-IP approach produce incomplete or inconsistent datasets.
• Browser fingerprinting: Anti-bot providers check for consistency between declared user-agent, accepted-language headers, and the IP's geolocation. A U.S. user-agent paired with a Japanese datacenter IP is an immediate red flag that triggers blocking or CAPTCHA challenges.

This is why rotating residential proxies are essential. Residential IPs come from real ISPs and real geographic locations, making them far less likely to trigger anti-bot heuristics than datacenter IPs. By rotating IPs per request and matching the IP's country to your target locale, you get both volume and consistency. For a deeper dive into proxy infrastructure, see our web scraping use case page.

Scraping Trustpilot Reviews: Parse __NEXT_DATA__ Instead of HTML

Trustpilot renders business-profile pages with Next.js, which embeds the full review payload as JSON inside a <script id="__NEXT_DATA__"> tag. Parsing this JSON is far more reliable than scraping HTML elements, which can change with any frontend deployment. This is the recommended approach to scrape Trustpilot reviews at scale.

Here's a Python example that paginates through a Trustpilot business profile, rotating IPs per page via ProxyHat's HTTP gateway on port 8080:

import requests
import json
import re
import time

PROXY = "http://user:pass@gate.proxyhat.com:8080"
PROXIES = {"http": PROXY, "https": PROXY}

def scrape_trustpilot_profile(slug, max_pages=10):
    """Scrape public reviews from a Trustpilot business profile."""
    base_url = f"https://www.trustpilot.com/review/{slug}"
    all_reviews = []

    for page in range(1, max_pages + 1):
        url = f"{base_url}?page={page}"
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/120.0.0.0 Safari/537.36"
            ),
            "Accept-Language": "en-US,en;q=0.9",
        }
        resp = requests.get(url, headers=headers, proxies=PROXIES, timeout=30)

        if resp.status_code == 200:
            match = re.search(
                r'<script id="__NEXT_DATA__" type="application/json">(.*?)</script>',
                resp.text, re.DOTALL
            )
            if match:
                data = json.loads(match.group(1))
                reviews = (
                    data.get("props", {})
                    .get("pageProps", {})
                    .get("reviews", [])
                )
                if not reviews:
                    break
                all_reviews.extend(reviews)
                print(f"Page {page}: {len(reviews)} reviews")
            else:
                print(f"Page {page}: __NEXT_DATA__ not found")
                break
        elif resp.status_code == 429:
            print(f"Page {page}: rate limited, backing off 5s")
            time.sleep(5)
            continue
        else:
            print(f"Page {page}: HTTP {resp.status_code}")
            break

        time.sleep(2)  # Polite delay between requests

    return all_reviews

reviews = scrape_trustpilot_profile("example.com", max_pages=5)
print(f"Total reviews collected: {len(reviews)}")

Each request exits through a different residential IP via gate.proxyhat.com:8080, so Trustpilot's per-IP rate limits don't accumulate against a single address. The __NEXT_DATA__ approach gives you structured fields — review title, body, star rating, author display name, and date — without fragile HTML parsing.

Scraping G2 Reviews: Heavier Anti-Bot, Residential IPs Required

G2 employs a more aggressive anti-bot posture than Trustpilot. Datacenter IPs are typically blocked within a handful of requests, and even residential IPs need realistic, consistent headers to pass inspection. The Cloudflare bot management guide outlines how providers use JavaScript challenges, TLS fingerprinting, and behavioral analysis to distinguish bots from humans. To scrape G2 reviews with a proxy, you need residential IPs plus disciplined header management.

Here's a Python example using country-targeted residential sessions:

import requests
import time

def scrape_g2_product_reviews(product_slug, max_pages=5):
    """Scrape public G2 product reviews with rotating residential proxies."""
    base = f"https://www.g2.com/products/{product_slug}/reviews"
    all_reviews = []

    for page in range(1, max_pages + 1):
        url = f"{base}?page={page}"
        # Country-targeted residential IP for consistent locale
        proxy = f"http://user-country-US:pass@gate.proxyhat.com:8080"
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/120.0.0.0 Safari/537.36"
            ),
            "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
            "Accept-Language": "en-US,en;q=0.9",
            "Accept-Encoding": "gzip, deflate, br",
            "Connection": "keep-alive",
            "Upgrade-Insecure-Requests": "1",
        }
        resp = requests.get(
            url, headers=headers,
            proxies={"https": proxy},
            timeout=30
        )

        if resp.status_code == 200:
            # G2 embeds review data in script tags and server-rendered HTML
            # Parse star distribution and individual reviews from the page
            print(f"Page {page}: {len(resp.text)} bytes received")
            # Add parsing logic based on current page structure
        elif resp.status_code == 403:
            print(f"Page {page}: blocked (403), retrying with new IP")
            time.sleep(3)
            continue
        elif resp.status_code == 429:
            print(f"Page {page}: rate limited, backing off 10s")
            time.sleep(10)
            continue
        else:
            print(f"Page {page}: HTTP {resp.status_code}")
            break

        time.sleep(3)  # Polite delay

    return all_reviews

reviews = scrape_g2_product_reviews("salesforce", max_pages=3)

Key practices for G2: keep headers consistent with a real browser session, use Accept-Language matching your IP's country, and add 2–5 second delays between requests. If you encounter a Cloudflare challenge page, back off and rotate to a new IP rather than retrying immediately. A target of 1–2 requests per second per domain is a reasonable baseline.

Google Business Reviews: Geo-Targeting for Consistent Locale

Google Business reviews scraping is heavily influenced by the hl and gl parameters, as well as the IP's geolocation. A request from a U.S. residential IP with hl=en&gl=US will surface English-language reviews in a U.S.-centric order. The same request from a French IP might prioritize French reviews or show a different review subset entirely. For consistent datasets, use country-targeted residential sessions.

For paginating through a single business's reviews, use a sticky session so the IP doesn't change mid-pagination:

import requests
import time

def scrape_google_business_reviews(place_id, country="US", lang="en", pages=5):
    """Scrape public Google Business reviews with geo-targeted sticky session."""
    # Sticky session keeps the same IP for up to 30 minutes
    session_id = "gb-reviews-001"
    proxy = (
        f"http://user-country-{country}-session-{session_id}:pass"
        f"@gate.proxyhat.com:8080"
    )

    all_reviews = []
    for page in range(pages):
        url = (
            f"https://www.google.com/maps/place/?q=place_id:{place_id}"
            f"&hl={lang}&gl={country.lower()}"
        )
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/120.0.0.0 Safari/537.36"
            ),
            "Accept-Language": f"{lang}-{country},{lang};q=0.9",
        }
        resp = requests.get(
            url, headers=headers,
            proxies={"https": proxy},
            timeout=30
        )

        if resp.status_code == 200:
            # Google Maps embeds review data in nested JS structures
            # Parse based on current page structure
            print(f"Page {page}: {len(resp.text)} bytes")
        elif resp.status_code == 429:
            print(f"Page {page}: rate limited, backing off")
            time.sleep(5)
            continue
        else:
            print(f"Page {page}: HTTP {resp.status_code}")
            break

        time.sleep(3)

    return all_reviews

# Scrape with U.S. residential sticky session
reviews = scrape_google_business_reviews(
    "ChIJN1t_tDeuEmsRUsoyG83frY4", country="US", lang="en", pages=5
)

The -country-US flag in the ProxyHat username routes your request through a U.S. residential IP, ensuring Google returns the U.S. locale's review ordering. The -session-abc123 flag keeps the same IP for up to 30 minutes — critical for paginating Google Business reviews, where mid-session IP changes can reset pagination state or trigger anti-bot checks. Check available proxy locations for country-level targeting options.

Node.js Example: Rotating IPs Across Review Pages

For JavaScript/TypeScript teams, here's a Node.js snippet using axios with ProxyHat's HTTP gateway to scrape Trustpilot reviews with per-request IP rotation:

const axios = require('axios');
const { HttpsProxyAgent } = require('https-proxy-agent');

const PROXY_URL = 'http://user:pass@gate.proxyhat.com:8080';
const agent = new HttpsProxyAgent(PROXY_URL);

async function scrapeTrustpilotReviews(slug, maxPages = 5) {
  const allReviews = [];

  for (let page = 1; page <= maxPages; page++) {
    const url = `https://www.trustpilot.com/review/${slug}?page=${page}`;

    try {
      const resp = await axios.get(url, {
        httpsAgent: agent,
        headers: {
          'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
          'Accept-Language': 'en-US,en;q=0.9',
        },
        timeout: 30000,
      });

      // Extract __NEXT_DATA__ JSON from the HTML
      const match = resp.data.match(
        /<script id="__NEXT_DATA__" type="application\/json">(.*?)<\/script>/s
      );

      if (match) {
        const data = JSON.parse(match[1]);
        const reviews = data.props?.pageProps?.reviews || [];
        if (reviews.length === 0) break;
        allReviews.push(...reviews);
        console.log(`Page ${page}: ${reviews.length} reviews`);
      } else {
        console.log(`Page ${page}: __NEXT_DATA__ not found`);
        break;
      }
    } catch (err) {
      console.error(`Page ${page}: ${err.message}`);
      break;
    }

    // Polite delay with jitter
    const delay = 2000 + Math.random() * 1000;
    await new Promise(r => setTimeout(r, delay));
  }

  return allReviews;
}

scrapeTrustpilotReviews('example.com', 5)
  .then(reviews => console.log(`Total: ${reviews.length} reviews`));

Proxy Type Comparison for Review Scraping

For most review-scraping workflows, residential proxies with per-request rotation are the sweet spot. Use sticky sessions when you need to paginate through a single profile without the IP changing mid-sequence. See our pricing for plan details.

Common Mistakes and Edge Cases

Inconsistent Headers

Sending a U.S. user-agent with Accept-Language: de-DE from a U.K. residential IP creates a fingerprint mismatch that anti-bot systems flag immediately. Always align your user-agent, Accept-Language, and proxy country.

Ignoring robots.txt

Before scraping any platform, check its robots.txt file. If a path is disallowed, respect it. Violating robots.txt can expose you to legal liability under the CFAA in the United States.

Scraping Too Fast

Even with perfect IP rotation, requesting 100 pages per second from a single platform will trigger behavioral analysis. Add 2–5 second delays between requests, and use jitter (randomized delays) to avoid detectable patterns. A target of 1–2 requests per second per domain is a reasonable starting point.

Not Handling Pagination Edge Cases

Trustpilot and G2 may return empty pages, redirect to a different URL, or serve a CAPTCHA page instead of a 403. Always check for these cases in your response parser rather than assuming a 200 status means valid data.

Collecting PII Unnecessarily

Review author names and avatars are technically public, but storing them in bulk datasets can create GDPR compliance issues, especially for EU-based reviewers. Consider hashing or anonymizing author identifiers in your storage layer.

Key Takeaways

• Residential proxies are essential for review scraping because platforms rate-limit by IP and personalize content by geography.
• Parse embedded JSON (like Trustpilot's __NEXT_DATA__) instead of brittle HTML selectors whenever possible.
• Match headers to IP locale — consistent user-agent, Accept-Language, and proxy country avoid fingerprint mismatches.
• Use sticky sessions (-session-abc123) for paginating a single profile; use per-request rotation for scraping many profiles.
• Respect ToS, robots.txt, and rate limits — prefer official APIs when available, and never harvest PII beyond what's necessary.

When to Use Official APIs Instead

Before building a scraper, check whether the platform offers an official API. Google provides the Google Business Profile API for managing your own business listings (though it does not expose competitor reviews). Trustpilot offers a consumer API for fetching public reviews of your own business. If an official API covers your use case, it will be more reliable and legally safer than scraping.

When no API exists or it's too restrictive for your needs, ProxyHat's residential proxy network gives you the infrastructure to collect public review data at scale. See our web scraping use case and SERP tracking pages for more implementation guidance. Full connection details are in the ProxyHat documentation.