How to Scrape Google Shopping Prices in 2026: A Practical Guide

If you're a pricing analyst or competitive-intelligence engineer, knowing what your competitors charge on Google Shopping is table stakes. But the Google Shopping price scraper landscape has shifted: the Content API for Shopping only returns data for merchants' own products, not competitor catalogs. That means competitive price monitoring requires parsing the tbm=shop SERP HTML or individual /shopping/product/ pages. This guide explains exactly how to scrape Google Shopping prices in 2026, including the selectors, proxy strategy, and anti-bot mitigations you need to keep your data pipeline running.

Why Scrape Google Shopping? The API-vs-HTML Trade-Off

Google offers two official programmatic paths for Shopping data:

• Content API for Shopping — lets a merchant manage and retrieve their own product feed, pricing, and inventory. It does not expose other sellers' products.
• Merchant Center API — same scope: your own account data only.

Neither API gives you a competitor's price for the same SKU. To collect Google Shopping product data across sellers, you must request the public SERP at https://www.google.com/search?tbm=shop&q=... and parse the returned HTML, or visit product detail pages at https://www.google.com/shopping/product/... and extract the seller-offer panel.

This is why teams build custom Google Shopping price scrapers — there's no official endpoint for cross-merchant competitive intelligence. The trade-off: you get the data you need, but you also inherit Google's anti-bot defenses.

Google's Anti-Bot Stack: What You're Up Against

Google has one of the most sophisticated anti-scraping systems on the web. When you scrape Google Shopping at any meaningful volume, you will encounter these layers:

1. The sorry/index Redirect

When Google's edge detects suspicious traffic from an IP, it redirects the request to https://www.google.com/sorry/index instead of returning SERP HTML. This page typically says "Our systems have detected unusual traffic" and may present a reCAPTCHA challenge. This is a soft block — the IP isn't permanently banned, but it's temporarily unusable for SERP requests.

2. reCAPTCHA Challenges

If the sorry page appears, Google may serve a reCAPTCHA v2 or v3 challenge. Solving it programmatically is fragile, expensive, and often violates Google's terms. The better strategy is to avoid triggering it in the first place by distributing requests across many residential IPs.

3. Per-IP Rate Limits

Google doesn't publish exact thresholds, but empirical testing shows that a single datacenter IP sending more than roughly 20–30 requests per minute to Shopping SERPs will hit the sorry redirect within minutes. Residential IPs tolerate slightly more, but the key insight is that concentration triggers blocks — not just raw volume.

4. Browser Fingerprinting

Google's anti-bot system also examines headers, TLS fingerprints, and behavioral patterns. A requests-based scraper with default headers is trivially identifiable. You'll need realistic User-Agent strings, Accept-Language headers matching your geo-target, and ideally TLS libraries that mimic real browsers.

Anti-Bot Layer	Trigger	Mitigation
sorry/index redirect	Too many requests from one IP	Rotate residential IPs; keep per-IP rate under ~20 req/min
reCAPTCHA challenge	Repeated suspicious patterns	Avoid triggering; use geo-matched residential proxies
Header/TLS fingerprinting	Non-browser request signatures	Use realistic headers; consider TLS-spoofing libraries
Geographic mismatch	US IP requesting German results	Match proxy country to gl / hl params

URL Patterns and CSS Selectors for Google Shopping

The Shopping SERP uses a dedicated query parameter: tbm=shop. The base URL pattern is:

https://www.google.com/search?tbm=shop&q=YOUR+QUERY&gl=us&hl=en

Key parameters:

• tbm=shop — forces the Shopping tab.
• q — your search query (URL-encoded).
• gl — geolocation country code (e.g., us, de, fr).
• hl — interface language (e.g., en, de, fr).
• num — results per page (max ~100, but Google may cap it).
• start — pagination offset (0, 20, 40, ...).

CSS Selectors for the Shopping SERP

Google's HTML structure changes frequently, but as of early 2026 these selectors are reliable:

• .sh-dgr__content — main container for each product result card.
• .sh-pr__product-results — wrapper around the full results grid.
• .a8Pemb — price text node (e.g., "$299.00").
• .tDxXle — product title link.
• span[style*="visibility"] — seller name in some layouts.
• span[aria-label*="Rated"] — star rating with review count.

For the product detail page at https://www.google.com/shopping/product/{product_id}, the seller-offer panel uses:

• sh-osd__offer — individual seller offer row.
• sh-osd__offer-price — per-seller price.
• sh-osd__offer-seller — seller name.
• sh-osd__offer-link — outbound link to the seller's product page.

Sample Truncated SERP Response

<div class="sh-dgr__content">
  <a class="tDxXle" href="/shopping/product/1234567890">
    Sony WH-1000XM5 Wireless Headphones
  </a>
  <span class="a8Pemb">$298.00</span>
  <span>Best Buy</span>
  <span aria-label="Rated 4.7 out of 5">★★★★☆ (1,243)</span>
</div>

Why Localized Prices Demand Residential Proxies with City-Level Geo

Google Shopping results are geo-dependent. A shopper in New York and a shopper in Berlin searching for the same product see different sellers, prices, currencies, and availability. This isn't just about language — Google uses the requesting IP's geographic location to determine which merchant offers to surface.

If you're a pricing analyst tracking competitor prices across markets, you need the requesting IP to originate from the target country — ideally the target city. A datacenter IP in Ashburn, Virginia will get US results, but it may also trigger Google's datacenter-IP heuristics faster than a residential IP would.

This is why residential proxies with city-level geo-targeting are essential for serious Google Shopping scraping. With ProxyHat, you can specify both country and city in the proxy username:

# US residential IP — see what American shoppers see
http://user-country-US:pass@gate.proxyhat.com:8080

# German residential IP in Berlin — see German sellers and EUR prices
http://user-country-DE-city-berlin:pass@gate.proxyhat.com:8080

# French residential IP in Paris
http://user-country-FR-city-paris:pass@gate.proxyhat.com:8080

Pair the proxy geo with the matching gl and hl URL parameters for maximum consistency:

• US: gl=us&hl=en + country-US proxy
• Germany: gl=de&hl=de + country-DE-city-berlin proxy
• France: gl=fr&hl=fr + country-FR-city-paris proxy

Mismatched combinations (e.g., a US proxy with gl=de) produce inconsistent results — Google may override the gl param based on IP location, or it may show a mix of both markets. Always align the two.

Worked Example: Scraping Google Shopping with Python and ProxyHat

Here's a complete Python example using requests and BeautifulSoup through the ProxyHat gateway. It scrapes a Shopping SERP for a product query, extracts title, price, seller, and rating from each result card, and handles the sorry-redirect soft block.

import requests
from bs4 import BeautifulSoup
import time
import random

# ProxyHat residential proxy — US IP for US Shopping results
proxy_url = "http://user-country-US:pass@gate.proxyhat.com:8080"
proxies = {"http": proxy_url, "https": proxy_url}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
}

def scrape_google_shopping(query, gl="us", hl="en", max_retries=3):
    base_url = "https://www.google.com/search"
    params = {
        "tbm": "shop",
        "q": query,
        "gl": gl,
        "hl": hl,
        "num": 20,
    }

    for attempt in range(max_retries):
        try:
            response = requests.get(
                base_url,
                params=params,
                headers=headers,
                proxies=proxies,
                timeout=30,
            )

            # Detect soft block — Google redirects to /sorry/index
            if "sorry/index" in response.url:
                print(f"Soft block detected on attempt {attempt + 1}. Waiting...")
                time.sleep(60 + random.uniform(10, 30))
                continue

            if response.status_code != 200:
                print(f"Status {response.status_code}, retrying...")
                time.sleep(10)
                continue

            soup = BeautifulSoup(response.text, "html.parser")
            results = []

            # Parse each product card
            for card in soup.select(".sh-dgr__content"):
                title_tag = card.select_one(".tDxXle")
                price_tag = card.select_one(".a8Pemb")
                seller_tag = card.find("span", string=True)
                rating_tag = card.select_one("span[aria-label*='Rated']")

                results.append({
                    "title": title_tag.get_text(strip=True) if title_tag else None,
                    "price": price_tag.get_text(strip=True) if price_tag else None,
                    "seller": seller_tag.get_text(strip=True) if seller_tag else None,
                    "rating": rating_tag.get("aria-label") if rating_tag else None,
                    "product_url": f"https://www.google.com{title_tag['href']}" if title_tag and title_tag.has_attr("href") else None,
                })

            return results

        except requests.RequestException as e:
            print(f"Request error: {e}")
            time.sleep(15)

    return []

# Example usage
results = scrape_google_shopping("Sony WH-1000XM5")
for r in results[:5]:
    print(f"{r['title']} | {r['price']} | {r['seller']} | {r['rating']}")

Sample output (truncated):

Sony WH-1000XM5 | $298.00 | Best Buy | Rated 4.7 out of 5
Sony WH-1000XM5 | $329.99 | Amazon | Rated 4.7 out of 5
Sony WH-1000XM5 | $279.00 | eBay | Rated 4.6 out of 5

Using SOCKS5 for Lower-Latency Connections

If you need SOCKS5 (useful for certain network configurations or when HTTP proxies are filtered), ProxyHat supports SOCKS5 on port 1080:

socks5://user-country-US:pass@gate.proxyhat.com:1080

In Python, use requests[socks] with PySocks:

proxy_url = "socks5://user-country-US:pass@gate.proxyhat.com:1080"
proxies = {"http": proxy_url, "https": proxy_url}

Pagination, Query Batching, and Soft-Block Detection

Pagination

Google Shopping paginates with the start parameter. Each page typically shows 20 results. To scrape deeper, increment start by 20:

for start in range(0, 100, 20):
    params["start"] = start
    results = scrape_google_shopping(query)
    time.sleep(random.uniform(3, 8))  # randomized delay
    if not results:
        break  # no more results or blocked

Don't go too deep — results beyond page 5–6 are often sparse or repeated. Most competitive pricing data lives on page 1–2.

Query Batching Strategy

If you're monitoring prices for 500 SKUs, don't fire all queries simultaneously. Batch them:

• Batch size: 10–20 concurrent queries max.
• Delay between batches: 30–60 seconds.
• Randomized per-request delay: 2–8 seconds between individual requests within a batch.
• Per-IP rotation: Use sticky sessions per query so pagination uses the same IP, but rotate between queries. The session- flag in the ProxyHat username controls this:

# Sticky session for one product's pagination
http://user-session-sku123-country-US:pass@gate.proxyhat.com:8080

# New session for the next product
http://user-session-sku456-country-US:pass@gate.proxyhat.com:8080

Detecting Soft Blocks Before a Hard CAPTCHA

The progression is: normal response → fewer results → sorry redirect → reCAPTCHA. Watch for these early signals:

1. Result count drops — if you normally get 20 results and suddenly get 3, Google may be throttling you.
2. Response time spikes — if latency jumps from 800ms to 5s+, you're being rate-limited.
3. Empty result containers — the page loads but .sh-dgr__content has zero matches.
4. Sorry redirect — response.url contains sorry/index. Back off immediately.

Implement a circuit breaker: if you see 2 consecutive soft-block signals, pause the scraper for 5–10 minutes and rotate to a fresh IP pool before resuming.

ProxyHat Setup and Configuration

Getting started with ProxyHat for Google Shopping scraping takes minutes. Here's the setup flow:

1. Create an account at dashboard.proxyhat.com and purchase a residential proxy plan that fits your volume needs. See pricing for current plans.
2. Get your credentials — your username and password from the dashboard.
3. Choose your geo-targeting — country-level for broad market data, city-level for precise local pricing. Browse available proxy locations.
4. Configure your scraper — use gate.proxyhat.com:8080 for HTTP or gate.proxyhat.com:1080 for SOCKS5, with geo and session flags in the username.

For more implementation details, check the ProxyHat documentation and our web scraping use case and SERP tracking guides.

Ethics, Terms of Service, and Compliance

Scraping Google Shopping sits in a legally gray area. Here's what you need to know:

Google's Terms of Service

Google's ToS prohibits automated querying of its services without permission. Violating ToS isn't necessarily illegal, but it can result in IP blocks, account suspension, and in extreme cases, legal action. The Computer Fraud and Abuse Act (CFAA) has been cited in scraping cases, though recent court rulings (e.g., hiQ Labs v. LinkedIn) have narrowed its applicability to public-data scraping.

GDPR and Data Privacy

Product prices and seller names are generally not personal data under GDPR. However, if your scraping collects review author names, user-generated content, or any personally identifiable information, GDPR compliance becomes relevant. Stick to public price and product data only.

Best Practices for Ethical Scraping

• Scrape public data only — prices, product titles, seller names visible without login.
• Respect rate limits — don't hammer Google's servers. 2–8 second delays between requests is reasonable.
• Don't bypass authentication — if data requires a Google login, don't scrape it.
• Consider official alternatives — for large-scale commercial use, Google's official Custom Search JSON API or licensed SERP API partners may be the compliant route, though they have their own rate limits and pricing.
• Cache aggressively — don't re-scrape the same query every 5 minutes. Most prices change daily, not hourly.

Key Takeaways

• The Content API for Shopping only returns your own products — competitive price monitoring requires HTML scraping of the tbm=shop SERP.
• Google's anti-bot stack includes the sorry/index redirect, reCAPTCHA, per-IP rate limits (~20–30 req/min), and browser fingerprinting.
• Use .sh-dgr__content, .a8Pemb, and .tDxXle selectors for the Shopping SERP; sh-osd__offer for product detail seller panels.
• Residential proxies with city-level geo (country-DE-city-berlin) plus matching gl/hl params are essential for accurate localized pricing.
• Use sticky sessions per query (session-sku123) for pagination, rotate IPs between queries, and implement a circuit breaker for soft-block detection.
• Scrape public data only, respect rate limits, and consider official SERP APIs for large-scale commercial use.

Frequently Asked Questions

What is a Google Shopping price scraper?

A Google Shopping price scraper is a tool or script that programmatically requests Google Shopping SERP pages (using the tbm=shop parameter) and parses the returned HTML to extract product titles, prices, seller names, and ratings. Because Google's Content API for Shopping only returns a merchant's own product data, competitive price monitoring requires scraping the public Shopping SERP or individual product detail pages at /shopping/product/.

Why do proxy users need residential IPs for Google Shopping scraping?

Google Shopping results are geo-dependent — a US shopper and a German shopper see different sellers, currencies, and prices for the same product. Datacenter IPs are also more aggressively flagged by Google's anti-bot system. Residential proxies with country and city-level geo-targeting (e.g., country-DE-city-berlin) ensure you see the same results a local shopper would, while reducing the risk of triggering the sorry/index redirect or reCAPTCHA challenges.

Which proxy type works best for scraping Google Shopping?

Residential proxies are the best choice for Google Shopping scraping. They use real ISP-assigned IP addresses, making them harder for Google's anti-bot system to detect. Datacenter proxies are cheaper but get blocked much faster — often within 20–30 requests per minute. Mobile proxies offer even higher trust scores but are more expensive and typically unnecessary for Shopping SERPs. For most pricing-monitoring use cases, residential proxies with city-level geo-targeting provide the right balance of reliability and cost.

How do you avoid blocks when scraping Google Shopping?

To avoid blocks: (1) rotate residential IPs with each query using session flags; (2) keep per-IP request rates under 20 per minute; (3) use randomized delays of 2–8 seconds between requests; (4) match proxy geo to gl/hl URL parameters; (5) use realistic browser headers including User-Agent and Accept-Language; (6) detect soft-block signals (result count drops, sorry redirects) early and back off with a circuit breaker before hitting a hard CAPTCHA.

Is scraping Google Shopping legal?

Scraping publicly visible price data from Google Shopping is generally legal under US law, but it violates Google's Terms of Service. The CFAA's applicability to public-data scraping has been narrowed by recent court rulings. In the EU, GDPR applies only if you collect personal data — product prices and seller names are typically not personal data. For large-scale commercial use, consider Google's official SERP API partners or the Custom Search JSON API as a compliant alternative.