How to Scrape Yelp Business Listings and Reviews in 2026: Public Data, PerimeterX, and Production Patterns

Legal caveat: This guide covers access to publicly available business information only. Before scraping Yelp, review Yelp's Terms of Service and robots.txt. In the United States, the Computer Fraud and Abuse Act (CFAA) and state laws may restrict automated access. In the European Union, the General Data Protection Regulation (GDPR) governs the processing of personal data, including review text tied to identifiable users. If your use case touches personal data, obtain legal advice and prefer Yelp's official Fusion API where it covers your needs. ProxyHat does not encourage unauthorized scraping.

If you build local-business datasets, competitive intelligence dashboards, or review-analysis pipelines, you have probably asked how to scrape Yelp business listings and reviews without getting blocked within the first 50 requests. Yelp is one of the richest sources of structured local-business data — names, addresses, categories, hours, ratings, and review text — but it is also one of the most aggressively defended. This guide walks through what is actually public, how Yelp's anti-bot stack works, and how to use rotating residential proxies to collect public business data responsibly and at scale.

What Is Publicly Accessible on Yelp Without Login

Yelp business profile pages at https://www.yelp.com/biz/<slug> are served to anonymous browsers without authentication. The page HTML and embedded JSON contain:

• Business identity: name, claimed-status, Yelp URL, phone, website link.
• Location: street address, city, state, postal code, neighborhood, geo-coordinates.
• Classification: primary category, sub-categories, tags, price range.
• Operational metadata: hours of operation (per day), special hours, attributes like "Good for Kids" or "Outdoor Seating".
• Aggregate signals: star rating (0–5), review count, photo count.
• Review text: the first few reviews are rendered server-side in the initial HTML and in an embedded __INITIAL_STATE__ JSON blob. Additional reviews load via an internal review_feed endpoint that returns JSON.

Content that is not accessible without login includes direct messages to business owners, full review history beyond what the public feed exposes, and some user-profile details. Yelp's official Fusion API exposes business search, details, and a reviews endpoint — but the reviews endpoint returns a maximum of 3 reviews per business, which is far below what the public page surfaces. That gap is the main reason developers turn to page scraping for review datasets.

Yelp's Anti-Bot Defenses: PerimeterX/HUMAN, TLS, and CAPTCHAs

Yelp uses HUMAN Security (formerly PerimeterX) as its primary bot-management layer. The defenses you will encounter include:

The _px3 cookie and sensor challenge

PerimeterX issues a _px3 cookie after a JavaScript "sensor" challenge that collects browser fingerprint signals — canvas rendering, WebGL, timing, device motion, and dozens of behavioral features. The sensor payload is POSTed to a PerimeterX endpoint, and a valid _px3 token is returned. Without it, requests are redirected to an interstitial challenge page or a CAPTCHA.

TLS fingerprinting (JA3/JA4)

PerimeterX inspects the TLS ClientHello to fingerprint your HTTP client. python-requests with default urllib3 settings produces a recognizable JA3 hash that differs from real Chrome or Firefox. Mismatches between your declared User-Agent and your TLS fingerprint are a strong bot signal.

IP-reputation scoring and CAPTCHA triggers

After as few as 5–20 requests from a datacenter IP range, Yelp typically returns a PerimeterX challenge or a 403. Datacenter ASN blocks from AWS, GCP, DigitalOcean, and Hetzner are flagged with very low trust scores. Residential IPs from major US ISPs carry much higher trust and survive longer before challenges appear.

Behavioral heuristics

Request velocity, navigation patterns (landing directly on /biz/ without a referrer), and missing headers (Accept-Language, sec-ch-ua, sec-fetch-*) all contribute to risk scoring. Real browsers send a predictable header cascade; scraping libraries often omit or misorder them.

Why Rotating Residential Proxies with US Geo Are Essential

Because PerimeterX weights IP reputation heavily, datacenter proxies fail fast on Yelp. Mobile and residential proxies map to real ISP allocations, so they inherit higher trust scores. For Yelp specifically, US residential proxies matter for two reasons:

1. Localized results: Yelp serves different business rankings and review snippets based on the viewer's geo. A request from a German IP may see different "nearby" businesses or even a localized UI. To get authentic US local-business data, route through US residential IPs.
2. IP-reputation survival: Rotating across a large US residential pool means each IP accrues minimal request history on Yelp, reducing the chance any single IP hits the challenge threshold.

ProxyHat residential proxies support country and city targeting via the username string, e.g. user-country-US-city-austin. For Yelp, city-level targeting is useful when you need results that reflect what a local user sees. See available proxy locations for the full country/city list.

Python Example: Fetching a Yelp Business Page via ProxyHat

This snippet uses Python requests with a ProxyHat residential proxy, US country targeting, and a sticky session. It fetches the business page, extracts the embedded __INITIAL_STATE__ JSON, and parses one truncated review object. The review_feed endpoint is mentioned in comments for pagination.

import requests
import json
import re
import time
import random

PROXY = "http://user-country-US-city-austin-session-biz001:pass@gate.proxyhat.com:8080"

proxies = {"http": PROXY, "https": PROXY}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept-Encoding": "gzip, deflate, br",
    "sec-ch-ua": '"Chromium";v="124", "Google Chrome";v="124", "Not.A/Brand";v="99"',
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": '"Windows"',
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "none",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1",
}

url = "https://www.yelp.com/biz/some-business-slug"

resp = requests.get(url, headers=headers, proxies=proxies, timeout=30)
print("Status:", resp.status_code)

if resp.status_code == 200 and "__INITIAL_STATE__" in resp.text:
    # Extract the embedded JSON blob
    match = re.search(r'__INITIAL_STATE__=({.*?});', resp.text)
    if match:
        state = json.loads(match.group(1))
        # Navigate to reviews (structure varies; adjust keys to current schema)
        biz = state.get("bizDetailsPageState", {}).get("bizDetails", {})
        reviews = biz.get("reviews", [])
        if reviews:
            r = reviews[0]
            print(json.dumps({
                "review_id": r.get("id"),
                "rating": r.get("rating"),
                "text": (r.get("comment", {}).get("text") or "")[:120] + "...",
                "date": r.get("localizedDate"),
                "author": r.get("author", {}).get("displayName"),
            }, indent=2))
else:
    print("Blocked or challenge page detected.")

# For pagination, the internal review_feed endpoint accepts:
#   GET /biz/<slug>/review_feed?rl=<offset>&sort_by=relevance_desc
# with the same proxy + headers. Pace requests 3–8 seconds apart.

For high-volume collection, you can use the ProxyHat SDK to manage rotation and sessions programmatically. Full connection details are in the ProxyHat documentation. For broader scraping patterns, see the web scraping use case page.

Node.js Example: SOCKS5 Proxy on Port 1080

For Node.js, use socks-proxy-agent with the ProxyHat SOCKS5 endpoint on port 1080. SOCKS5 is useful when you need TCP-level proxying for libraries that do not support HTTP CONNECT cleanly.

const { SocksProxyAgent } = require('socks-proxy-agent');
const fetch = require('node-fetch');

const proxyUrl = 'socks5://user-country-US-session-node01:pass@gate.proxyhat.com:1080';
const agent = new SocksProxyAgent(proxyUrl);

const headers = {
  'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 '
                + '(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',
  'Accept': 'text/html,application/xhtml+xml;q=0.9,*/*;q=0.8',
  'Accept-Language': 'en-US,en;q=0.9',
  'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124"',
  'sec-fetch-dest': 'document',
  'sec-fetch-mode': 'navigate',
  'sec-fetch-site': 'none',
};

(async () => {
  const res = await fetch('https://www.yelp.com/biz/some-business-slug', {
    agent,
    headers,
  });
  console.log('Status:', res.status);
  const html = await res.text();
  const m = html.match(/__INITIAL_STATE__=({.*?});/);
  if (m) {
    const state = JSON.parse(m[1]);
    const reviews = state?.bizDetailsPageState?.bizDetails?.reviews || [];
    console.log('Reviews found:', reviews.length);
    if (reviews[0]) {
      console.log('First rating:', reviews[0].rating);
    }
  } else {
    console.log('No __INITIAL_STATE__ found — possible block.');
  }
})();

Pagination, Rate-Limit Pacing, Sticky Sessions, and User-Agent Rotation

Pagination via review_feed

Yelp loads additional reviews through an internal JSON endpoint, typically /biz/<slug>/review_feed?rl=&sort_by=relevance_desc. Each response returns a batch of review objects. Increment rl to page through results. Not all businesses expose the same number of reviews publicly — some cap around 2,000–3,000 reviews in the public feed.

Rate-limit pacing

PerimeterX triggers on velocity. A safe starting pace is 1 request every 5–10 seconds per session/IP, with randomized jitter. If you run multiple concurrent workers, give each a distinct sticky session so each IP maintains a consistent identity. Spreading 10 workers across 10 sticky residential IPs at 8-second intervals yields roughly 7,500 requests/day — enough for a focused business dataset without aggressive risk.

Sticky sessions for continuity

Use the -session- flag in the ProxyHat username to pin a residential IP for the session lifetime. This is critical for Yelp because PerimeterX correlates the _px3 cookie with the IP that earned it. Rotating IP mid-session invalidates the cookie and forces a new sensor challenge. A pattern like user-country-US-session-biz- keeps one IP per business scrape.

Rotating User-Agents to lower fingerprint risk

Rotate among 5–10 current, real Chrome/Firefox User-Agent strings that match your TLS fingerprint. If you use requests, consider curl_cffi or tls-client to impersonate Chrome's TLS ClientHello. Mismatched UA + TLS is one of the most common reasons scrapers get blocked even with residential IPs. Keep the full header set consistent — a Chrome UA should send sec-ch-ua headers; a Firefox UA should not.

Common Mistakes and Edge Cases

• Using datacenter IPs: AWS/GCP IPs are flagged within a handful of requests. Use residential.
• Missing sec-fetch headers: Browsers always send sec-fetch-* on navigations. Omitting them is a bot signal.
• Direct deep links with no referrer: Landing on /biz/ from nowhere looks suspicious. Consider setting a realistic Referer like https://www.yelp.com/search?find_desc=....
• Ignoring the _px3 cookie: If you get challenged once, that IP may be burned for hours. Rotate immediately on a 403 or challenge response.
• Scraping review author profiles: User profile pages often require login and contain personal data. Avoid them — they add GDPR risk and provide little business value.
• Assuming the __INITIAL_STATE__ schema is stable: Yelp changes key names periodically. Build defensive parsers with .get() and log missing keys.

Comparison: Scraping vs Fusion API

Aspect	Public Page Scraping	Fusion API
Reviews per business	Hundreds to thousands (public feed)	Max 3 reviews
Business fields	Rich (hours, attributes, photos)	Core fields only
Rate limits	Self-managed; PerimeterX gated	Official quota (e.g. 5,000 calls/day on standard tier)
Legal risk	Higher — ToS/CFAA/GDPR exposure	Lower — licensed access
Reliability	Breaks when schema or anti-bot changes	Stable, versioned
Cost	Proxy bandwidth	API subscription + per-call fees

If the Fusion API's 3-review cap and core business fields satisfy your use case, use it. It is the legally safer path. See ProxyHat pricing for proxy bandwidth costs if you proceed with scraping.

Ethical Scraping and When to Use Official APIs

Scraping public business data can be legitimate, but the line between public data and protected data is not always obvious. Follow these principles:

• Respect robots.txt. Check https://www.yelp.com/robots.txt before crawling. If a path is disallowed, do not scrape it.
• Avoid login-walled content. Anything behind authentication is off-limits — it signals Yelp intended it to be private.
• Minimize personal data collection. Review text is tied to usernames. Under GDPR, personal data includes online identifiers. If you store reviews, consider pseudonymizing author names or storing only aggregate sentiment.
• Rate-limit responsibly. Do not hammer endpoints. 1 request per 5+ seconds per IP is a reasonable floor.
• Prefer the Fusion API where possible. If you only need business name, address, rating, and 3 reviews, the API is faster, cheaper, and legally safer.
• Do not republish raw scraped content. Yelp's content is copyrighted. Use extracted data for analysis, not republication.

For SERP and local-search monitoring workflows that complement Yelp data, see the SERP tracking use case.

Key Takeaways

• Yelp business pages expose rich public data (name, address, hours, rating, review text) without login, but the Fusion API caps reviews at 3 per business.
• PerimeterX/HUMAN defends Yelp with the _px3 sensor challenge, TLS fingerprinting, and IP-reputation scoring. Datacenter IPs get blocked within ~5–20 requests.
• Rotating US residential proxies with city targeting and sticky sessions are the foundation of reliable Yelp scraping.
• Parse the embedded __INITIAL_STATE__ JSON for server-rendered reviews; paginate via the internal review_feed endpoint.
• Pace at 5–10 seconds per request per IP, rotate User-Agents with matching TLS fingerprints, and rotate IPs on any 403 or challenge.
• Prefer the official Fusion API when its field coverage and 3-review cap meet your needs — it is the legally safer path.

FAQ

What is a Yelp review scraper?

A Yelp review scraper is a program or service that extracts publicly visible review text, ratings, and metadata from Yelp business pages. It typically uses HTTP clients to fetch business profile HTML or internal JSON endpoints, parses the embedded data, and stores it in a structured format like CSV or JSON. Scrapers must navigate anti-bot defenses and respect legal constraints.

Why does scraping Yelp matter for proxy users?

Yelp aggressively blocks datacenter IPs using PerimeterX/HUMAN, which scores IP reputation. Proxy users need rotating residential IPs to mimic real ISP traffic and avoid challenges. Without proxies, most scraping attempts fail within a handful of requests. Proxies also enable geo-targeting so results match what local users see.

Which proxy type works best for scraping Yelp?

Rotating residential proxies with US geo-targeting work best for Yelp. Residential IPs inherit trust from real ISPs, surviving far longer than datacenter ranges. City-level targeting (e.g. -country-US-city-austin) ensures localized business results. Sticky sessions keep one IP per scrape so the PerimeterX cookie stays valid. Mobile proxies also work but are slower and costlier.

How do you avoid blocks when scraping Yelp?

Pace requests at 5–10 seconds per IP, use sticky residential sessions, rotate among current Chrome/Firefox User-Agents with matching TLS fingerprints, send complete sec-fetch headers, and rotate IPs immediately on any 403 or challenge page. Avoid login-walled content, respect robots.txt, and prefer the Fusion API when it covers your data needs.

Can you scrape Yelp reviews without violating the law?

Scraping purely public business data may be permissible in some jurisdictions, but Yelp's Terms of Service and laws like the US CFAA and EU GDPR create real risk. Review text tied to usernames is personal data under GDPR. Always consult legal counsel, minimize personal data collection, and prefer the official Fusion API for licensed access where possible.