How to Scrape Yelp Business Listings and Reviews in 2026: Public Data, PerimeterX, and Proxies

Important legal caveat: This guide covers access to publicly available business data only. Before scraping Yelp, review their Terms of Service. Unauthorized automated access may violate the Computer Fraud and Abuse Act (CFAA) in the US and the GDPR in the EU, particularly when personal data (reviewer names, photos) is involved. Always prefer official APIs where they meet your needs, and consult legal counsel for your specific use case.

If you're building local-business datasets, review aggregations, or competitive intelligence pipelines, you've probably hit Yelp's anti-bot wall. Learning how to scrape Yelp business listings and reviews in 2026 means navigating PerimeterX/HUMAN challenges, TLS fingerprinting, and IP-reputation scoring — all while staying within public-data boundaries. This guide walks through what's accessible, what blocks you, and how to do it responsibly with rotating residential proxies.

How to Scrape Yelp Business Listings and Reviews in 2026: The Landscape

Yelp remains one of the richest sources of local business data: categories, hours, ratings, review text, photos, and geographic coordinates. But the platform has progressively hardened its defenses. Datacenter IPs get CAPTCHA challenges after as few as 3–5 requests. The _px3 cookie from PerimeterX (now HUMAN) blocks automated browsers and HTTP clients alike. And the official Yelp Fusion API caps reviews at three per business — far below what most dataset builders need.

The result: developers who need comprehensive review datasets must either work within the API's limits or carefully access public web pages using proxies that mimic real user traffic. This guide focuses on the latter, with an emphasis on legitimate public-data access and compliance.

What's Publicly Accessible Without Login

Yelp business pages at /biz/<slug> are publicly viewable without authentication. The following data elements are visible to any browser visitor:

• Business name, address, phone — displayed in the page header and structured data.
• Star rating and review count — aggregate numbers shown prominently.
• Categories — e.g., "Italian, Pizza, Restaurants" as clickable tags.
• Operating hours — weekly schedule in a structured block.
• Review text, author display name, rating, date — the first ~10 reviews are rendered server-side.
• Photos — user-uploaded and business-owner images.

What is not accessible without login includes: full review history beyond the initial page load, direct messages, bookmark lists, and any content behind a login wall. Scraping login-walled content crosses into clearer ToS violation territory and should be avoided.

Fusion API vs. Web Scraping: A Comparison

If the three-review cap is sufficient for your use case — say, sentiment sampling or rating verification — the Fusion API is the safer, faster path. For comprehensive review datasets, web scraping public pages is the only option, but it carries greater technical and legal complexity.

Yelp's Anti-Bot Defenses in 2026

Yelp employs a layered defense stack. Understanding each layer is essential before writing any scraping code.

PerimeterX / HUMAN Challenge

The _px3 cookie is the centerpiece of Yelp's bot detection. PerimeterX (acquired by HUMAN) issues a JavaScript sensor challenge that collects browser fingerprint data — canvas rendering, WebGL parameters, timing metrics, and device characteristics. If the sensor data doesn't match expected patterns, the request is blocked with a CAPTCHA or a 403 response.

For HTTP-only clients (like Python requests), the sensor challenge never executes, so the _px3 cookie is never generated. This means raw HTTP requests to Yelp pages typically fail within a handful of attempts from datacenter IPs.

TLS Fingerprinting

Yelp's edge infrastructure inspects TLS ClientHello fingerprints. Python's requests library uses urllib3's default TLS stack, which has a recognizable JA3/JA4 fingerprint that differs from real browsers. This alone can trigger blocks even before PerimeterX evaluates the request. Tools like curl-impersonate or libraries that wrap a real browser TLS stack can mitigate this, but they add complexity.

IP Reputation Scoring

Datacenter IP ranges from AWS, GCP, Azure, and major hosting providers are flagged with high bot-probability scores. A datacenter IP might survive 3–5 requests before receiving a CAPTCHA. Residential IPs, by contrast, carry the reputation of an ISP-assigned address and blend into normal traffic patterns, dramatically increasing request survival rates.

Why Rotating Residential Proxies Are Essential

For scraping Yelp, residential proxies serve two critical purposes: IP-reputation survival and geo-localized results.

IP reputation: Residential IPs are assigned by ISPs to real households. They carry low bot-probability scores in PerimeterX's reputation database. A rotating residential pool lets you distribute requests across hundreds or thousands of IPs, keeping per-IP request volume low enough to avoid triggering rate-based blocks.

Geo-localization: Yelp serves different business results based on the requester's geographic location. If you're scraping businesses in Austin, TX, routing through a US-based residential IP — ideally one geo-targeted to the relevant metro — yields more accurate and complete local results. ProxyHat supports country- and city-level targeting via the username parameter.

Using ProxyHat's geo-targeted residential pool, you can pin requests to specific US cities:

http://user-country-US-city-austin:pass@gate.proxyhat.com:8080

This produces results as seen by a real Austin resident, which matters for local search and business discovery workflows.

Python Implementation: Parsing Reviews and Initial State

Below is a Python example using requests through ProxyHat's residential gateway. It fetches a Yelp business page, extracts the embedded __INITIAL_STATE__ JSON blob (which contains structured business and review data), and parses a truncated review object.

import requests
import json
import re
import time
import random

# ProxyHat residential proxy with US geo-targeting
proxy_url = "http://user-country-US-city-austin:pass@gate.proxyhat.com:8080"

proxies = {
    "http": proxy_url,
    "https": proxy_url,
}

headers = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/124.0.0.0 Safari/537.36"
    ),
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept-Encoding": "gzip, deflate, br",
}

def fetch_yelp_biz(slug, session_id=None):
    """Fetch a Yelp business page and extract __INITIAL_STATE__."""
    # Use sticky session for pagination continuity
    if session_id:
        proxy = f"http://user-country-US-city-austin-session-{session_id}:pass@gate.proxyhat.com:8080"
    else:
        proxy = "http://user-country-US-city-austin:pass@gate.proxyhat.com:8080"

    p = {"http": proxy, "https": proxy}
    url = f"https://www.yelp.com/biz/{slug}"

    resp = requests.get(url, headers=headers, proxies=p, timeout=30)
    if resp.status_code != 200:
        print(f"Blocked or error: {resp.status_code}")
        return None

    # Extract __INITIAL_STATE__ JSON blob
    match = re.search(
        r'<script[^>]*>window\.__INITIAL_STATE__\s*=\s*({.*?});?</script>',
        resp.text,
        re.DOTALL
    )
    if not match:
        print("Could not find __INITIAL_STATE__")
        return None

    state = json.loads(match.group(1))
    return state

def parse_reviews(state, max_reviews=5):
    """Extract truncated review objects from initial state."""
    reviews = []
    # Navigate the nested state structure — path varies by page version
    biz_data = state.get("bizDetailsPageLayout", {}).get("bizDetailsPage", {})
    review_list = biz_data.get("reviews", [])

    for r in review_list[:max_reviews]:
        review = {
            "author": r.get("author", {}).get("displayName", ""),
            "rating": r.get("rating", None),
            "date": r.get("localizedDate", ""),
            "text": r.get("comment", {}).get("text", "")[:200],  # truncated
        }
        reviews.append(review)

    return reviews

# Example usage
if __name__ == "__main__":
    slug = "uchi-austin"
    state = fetch_yelp_biz(slug, session_id="yelp-001")
    if state:
        reviews = parse_reviews(state)
        for i, r in enumerate(reviews, 1):
            print(f"Review {i}: {r['rating']} stars by {r['author']} on {r['date']}")
            print(f"  Text: {r['text'][:100]}...")
            print()

    # Pace requests: 2-4 second random delay between fetches
    time.sleep(random.uniform(2, 4))

Note: The exact JSON path inside __INITIAL_STATE__ changes between Yelp page deployments. Treat the parsing logic as a starting point and inspect the actual structure for your target pages. The [:200] truncation on review text is intentional — it limits stored data volume and reduces personal-data exposure.

Alternative: Yelp's Review Feed JSON

Some Yelp pages expose a review feed endpoint that returns JSON directly. You can find the URL pattern in network traffic when scrolling the reviews section. It typically looks like /biz_photos/biz_id/review_feed or similar. Requesting it with the same proxy and headers can return structured review data without HTML parsing, but the endpoint is also protected by PerimeterX.

Node.js Example: SOCKS5 Proxy on Port 1080

For Node.js environments, ProxyHat's SOCKS5 gateway on port 1080 works well with libraries like socks-proxy-agent:

const { SocksProxyAgent } = require('socks-proxy-agent');
const fetch = require('node-fetch');

const agent = new SocksProxyAgent(
  'socks5://user-country-US-city-austin:pass@gate.proxyhat.com:1080'
);

const headers = {
  'User-Agent':
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ' +
    'AppleWebKit/537.36 (KHTML, like Gecko) ' +
    'Chrome/124.0.0.0 Safari/537.36',
  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
  'Accept-Language': 'en-US,en;q=0.9',
};

async function fetchYelpBiz(slug) {
  const url = `https://www.yelp.com/biz/${slug}`;
  const resp = await fetch(url, { agent, headers, timeout: 30000 });

  if (!resp.ok) {
    console.error(`HTTP ${resp.status} — likely blocked`);
    return null;
  }

  const html = await resp.text();
  const match = html.match(
    /window\.__INITIAL_STATE__\s*=\s*({[\s\S]*?});<\/script>/
  );

  if (!match) {
    console.error('No __INITIAL_STATE__ found');
    return null;
  }

  const state = JSON.parse(match[1]);
  return state;
}

(async () => {
  const state = await fetchYelpBiz('uchi-austin');
  if (state) {
    console.log('State keys:', Object.keys(state));
  }
})();

SOCKS5 can offer slightly better performance than HTTP CONNECT tunneling in some environments, and it handles HTTPS traffic cleanly without double-encryption concerns. See ProxyHat's documentation for full SOCKS5 configuration details.

Pagination, Rate Limits, and Session Continuity

Scraping Yelp at any meaningful scale requires careful request pacing and session management. Here's what works in practice:

Sticky Sessions for Pagination

When paginating through reviews for a single business, you want all requests to come from the same IP. If the IP rotates mid-sequence, PerimeterX may flag the session as suspicious. ProxyHat's -session- flag pins requests to a single IP for the session duration:

http://user-country-US-session-yelp-biz-001:pass@gate.proxyhat.com:8080

Use a unique session ID per business (e.g., yelp-biz-<slug>) and rotate to a new session when moving to the next business.

Rate-Limit Pacing

Avoid bursting requests. A practical pacing strategy:

• 2–4 second random delay between requests within a sticky session.
• 10–15 second pause when rotating to a new business/session.
• Max ~100 requests per IP per hour before rotating to a fresh residential IP.
• Cap concurrency at 5–10 parallel sessions to avoid pattern detection.

These numbers are conservative starting points. Monitor your success rate and adjust. If you see 403s or CAPTCHA pages increasing, slow down or rotate IPs more aggressively.

Rotating User-Agents

Even with residential IPs, sending the same User-Agent on every request creates a detectable pattern. Rotate among 5–10 current, real browser User-Agent strings. Match the UA to other headers — a Chrome UA should pair with Chrome-appropriate Accept and sec-ch-ua headers. Inconsistent header sets are a fingerprinting red flag.

import random

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15",
]

def get_headers():
    return {
        "User-Agent": random.choice(USER_AGENTS),
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        "Accept-Language": "en-US,en;q=0.9",
        "Accept-Encoding": "gzip, deflate, br",
        "Connection": "keep-alive",
    }

Common Mistakes and Edge Cases

• Ignoring robots.txt: Yelp's robots.txt may disallow certain paths. Check it before scraping and respect directives for blocked paths. See yelp.com/robots.txt.
• Storing full review text with reviewer names: Review text combined with a display name can constitute personal data under GDPR. Consider hashing or anonymizing reviewer identifiers and truncating text.
• Bursting requests on session rotation: When a sticky session ends and you start a new one, add a delay. Immediate bursting to a new IP still creates a traffic pattern.
• Not handling CAPTCHA responses: Detect 403 status codes and CAPTCHA HTML patterns early. Log them, back off, and rotate to a fresh IP rather than retrying immediately.
• Scraping during peak hours: Yelp's anti-bot sensitivity may increase during high-traffic periods. Off-hour scraping (late night US time) often has higher success rates.
• Assuming page structure is stable: Yelp A/B tests page layouts. The __INITIAL_STATE__ structure can change weekly. Build parsers defensively with fallback paths.

ProxyHat Setup and Configuration

Getting started with ProxyHat for Yelp scraping is straightforward. Configure your proxy authentication in the ProxyHat dashboard, then use the gateway details below.

For broader web scraping workflows, see our web scraping use case guide. If you're also tracking search rankings for local businesses, our SERP tracking guide covers complementary techniques.

Ethical Scraping and When to Use Official APIs

Scraping public data is a gray area legally and ethically. Here's a framework for making responsible decisions:

Prefer the Fusion API When Possible

If the Yelp Fusion API's three-review cap and 5,000 calls/day limit meet your needs, use it. It's sanctioned, stable, and carries minimal legal risk. Register for a free API key at the Yelp Developer Portal. The API covers business search, phone lookup, transaction details, and limited reviews — sufficient for many applications.

Respect Personal Data Boundaries

Reviews contain personal data: reviewer names, photos, and location history. Under GDPR, processing this data requires a lawful basis. For research or aggregation purposes, consider:

• Anonymizing or pseudonymizing reviewer identifiers.
• Truncating review text rather than storing full content.
• Not cross-referencing reviewer data with other datasets to re-identify individuals.
• Implementing data retention limits (e.g., delete after 90 days).

Avoid Login-Walled Content

Content behind Yelp's login wall — full review history, direct messages, user profiles — should not be scraped. Accessing it requires authentication, which implies accepting the ToS, and automated access to authenticated content is more clearly a ToS violation and potential CFAA concern.

Respect robots.txt and Rate Limits

Check robots.txt before scraping and honor disallow directives. Even if a path isn't disallowed, keep request rates reasonable. Aggressive scraping that degrades service for other users is both unethical and more likely to trigger legal action.

When to Seek Legal Counsel

If your use case involves commercial resale of scraped data, large-scale collection (millions of records), or cross-border data transfers, consult a lawyer familiar with CFAA, GDPR, and CCPA. The legal landscape around web scraping continues to evolve, and what's permissible for research may not be for commercial products.

Key Takeaways

Yelp's public business pages contain rich data — ratings, categories, hours, and up to ~10 reviews per page — but PerimeterX/HUMAN anti-bot defenses, TLS fingerprinting, and IP-reputation scoring make datacenter-based scraping unreliable. Rotating residential proxies with US geo-targeting (e.g., -country-US-city-austin) are essential for both survival and result accuracy. Use sticky sessions for pagination, pace requests at 2–4 second intervals, and rotate User-Agents to reduce fingerprint risk. Always prefer the Fusion API where its three-review cap is sufficient, and handle reviewer data carefully under GDPR. Check robots.txt, avoid login-walled content, and consult legal counsel for commercial use cases.

FAQ

What is the best way to scrape Yelp business listings and reviews in 2026?

The best approach combines rotating residential proxies with US geo-targeting, sticky sessions for pagination continuity, and parsing the embedded __INITIAL_STATE__ JSON blob from public business pages. Avoid datacenter IPs — they get CAPTCHA challenges within 3–5 requests. If you only need up to three reviews per business, the official Yelp Fusion API is the safer, simpler path.

Why does Yelp block datacenter proxy IPs so quickly?

Yelp uses PerimeterX/HUMAN for bot detection, which scores IP reputation. Datacenter IP ranges from AWS, GCP, and Azure carry high bot-probability scores. Residential ISP-assigned IPs blend into normal user traffic and survive significantly longer. Additionally, TLS fingerprinting detects non-browser HTTP clients, compounding the block rate for datacenter requests.

Which proxy type works best for scraping Yelp?

Rotating residential proxies with country- and city-level geo-targeting work best. US geo-targeting (e.g., -country-US-city-austin) produces locally accurate business results and carries strong IP reputation. Use sticky sessions (-session-<id>) when paginating through a single business's reviews to maintain IP continuity and avoid session-level blocks.

How do you avoid CAPTCHAs when scraping Yelp?

Use residential proxies, rotate User-Agents among 5–10 real browser strings, pace requests at 2–4 second random intervals, cap concurrency at 5–10 sessions, and limit each IP to ~100 requests per hour. Detect 403 responses and CAPTCHA HTML early, then back off and rotate to a fresh IP. Off-hour scraping (late night US time) often yields higher success rates.

Is scraping Yelp reviews legal?

Scraping publicly accessible data is generally permissible but exists in a legal gray area. Yelp's Terms of Service prohibit unauthorized automated access, and the CFAA has been applied to scraping cases. Reviewer data may constitute personal data under GDPR. Always prefer the Fusion API where possible, avoid login-walled content, respect robots.txt, and consult legal counsel for commercial use cases.