Legal note: This guide covers collecting publicly visible Amazon search results for legitimate rank-tracking purposes. In the US, the Computer Fraud and Abuse Act (CFAA) can apply to unauthorized access; stick to public data and respect Amazon's Terms of Service. In the EU, the GDPR governs personal data — search result pages generally contain no personal data, but be mindful if you log user-generated content. Always consult legal counsel for your specific use case.
Why Amazon Keyword Rank Tracking with Proxies Matters
Amazon's search engine — internally called A9 (now part of Amazon's broader search and recommendation systems) — is fundamentally different from Google's. Ranking is driven by a combination of text relevance and sales velocity. A product that converts well for a keyword climbs the ranks, which in turn drives more impressions and sales, creating a feedback loop. For sellers, this means knowing your organic position for a keyword, on a specific marketplace, for a specific ASIN, is mission-critical.
Unlike Google SERPs, Amazon results are heavily commercial: every page mixes organic results with Sponsored placements. If you don't separate the two, your rank-tracking data is polluted. An ASIN might appear in position 1 as a Sponsored ad while sitting at position 12 organically — those are very different signals for listing optimization.
Amazon also localizes results per marketplace. amazon.com, amazon.de, amazon.co.uk, and amazon.co.jp each return different rankings for the same keyword. To track accurately, you need to request pages from an IP in the target country. That's where residential proxies come in.
How Amazon's SERP Is Structured
When you search Amazon for a keyword, the results page returns a list of product cards. Each card is a <div> with the attribute data-component-type="s-search-result". The ASIN is stored in the data-asin attribute. Sponsored listings include a visible "Sponsored" label (often in a <span> with class containing s-label-popover-default or similar, but the text content is the reliable anchor).
Here's a simplified view of what one result card looks like:
<div data-component-type="s-search-result" data-asin="B0CXYZ1234" class="s-result-item">
<div class="puis-card-container">
...
<span class="a-color-secondary">Sponsored</span>
<h2><a href="/dp/B0CXYZ1234/...">Product Title</a></h2>
...
</div>
</div>To compute the organic rank, you iterate all result cards, skip those marked Sponsored, and assign a 1-based position to the remaining ones. The first organic result is rank 1, the second is rank 2, and so on.
Sponsored vs Organic: Why Separation Matters
Sponsored placements are paid ads. They appear because a seller bid on that keyword through Amazon Advertising. Organic rank is determined by the A9 algorithm based on relevance, conversion rate, review quality, and sales history. If you track both together, you can't tell whether a rank change came from increased ad spend or genuine organic improvement. For a keyword rank tracker, you should record both separately — Sponsored position and organic position — so sellers can see the full picture.
Why You Need Residential Proxies with Country Geo-Targeting
Amazon's anti-bot systems are aggressive. If you send dozens of search requests from a single datacenter IP, you'll hit CAPTCHAs and temporary bans within minutes — often within 50–100 requests. Datacenter IPs are trivially detectable because they belong to known cloud provider ASN ranges. Residential proxies route traffic through real ISP-assigned home connections, making requests look like genuine shoppers.
Amazon also localizes results by the IP's country. If you search amazon.de from a US IP, you may get redirected or see different results than a German shopper. To get accurate rank data for amazon.de, you need a German residential IP. For amazon.com, a US residential IP. For amazon.co.jp, a Japanese residential IP.
With ProxyHat, you control the country via the username parameter. For Amazon US, you use -country-US; for Amazon Germany, -country-DE. For stable pagination across multiple result pages (page 1 through 5), you also want a sticky session so the same IP handles all requests in the batch — otherwise Amazon may serve inconsistent results mid-sequence.
| Marketplace | Domain | ProxyHat Country Flag | Example Username |
|---|---|---|---|
| United States | amazon.com | -country-US | user-country-US-session-amz1 |
| Germany | amazon.de | -country-DE | user-country-DE-session-amz1 |
| United Kingdom | amazon.co.uk | -country-GB | user-country-GB-session-amz1 |
| Japan | amazon.co.jp | -country-JP | user-country-JP-session-amz1 |
| France | amazon.fr | -country-FR | user-country-FR-session-amz1 |
Setting Up ProxyHat for Amazon Rank Tracking
ProxyHat provides residential, mobile, and datacenter proxies through a single gateway at gate.proxyhat.com. The HTTP port is 8080 and the SOCKS5 port is 1080. All geo-targeting and session control flags go in the username field, separated by hyphens.
Here's the connection format:
# HTTP proxy — US residential with sticky session
http://user-country-US-session-amz_keyword1:pass@gate.proxyhat.com:8080
# SOCKS5 proxy — Germany residential with sticky session
socks5://user-country-DE-session-amz_keyword1:pass@gate.proxyhat.com:1080The -session- flag creates a sticky IP. Use a unique session ID per keyword batch so each keyword gets a fresh IP but all pages (1–5) for that keyword share the same IP. This mimics a real shopper browsing through multiple pages of results.
Code Example 1: Raw Proxy with curl_cffi
curl_cffi is a Python library that wraps libcurl with browser TLS fingerprinting. It's lighter than a full browser automation stack and fast enough for high-volume scraping. Here's how to fetch Amazon search results with ProxyHat residential proxies:
import asyncio
from curl_cffi.requests import AsyncSession
from urllib.parse import quote_plus
import json
import time
PROXY_USER = "user-country-US-session-amz_{kw}"
PROXY_PASS = "your_password"
PROXY_HOST = "gate.proxyhat.com"
PROXY_PORT = 8080
async def fetch_amazon_page(keyword: str, page: int, session_id: str) -> str:
"""Fetch one Amazon search results page via ProxyHat residential proxy."""
url = f"https://www.amazon.com/s?k={quote_plus(keyword)}&page={page}"
proxy_url = f"http://{PROXY_USER.format(kw=session_id)}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
async with AsyncSession(impersonate="chrome") as s:
resp = await s.get(
url,
proxies={"http": proxy_url, "https": proxy_url},
timeout=30,
headers={
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
},
)
if resp.status_code == 200:
return resp.text
elif resp.status_code == 503:
raise Exception(f"CAPTCHA or block detected (HTTP {resp.status_code})")
else:
raise Exception(f"Unexpected status: {resp.status_code}")
async def fetch_keyword_pages(keyword: str, max_pages: int = 5) -> list[str]:
"""Fetch pages 1 through max_pages for a keyword with retries."""
session_id = keyword.replace(" ", "_")[:20]
pages = []
for page in range(1, max_pages + 1):
attempt = 0
while attempt < 3:
try:
html = await fetch_amazon_page(keyword, page, session_id)
pages.append(html)
break
except Exception as e:
attempt += 1
wait = 2 ** attempt
print(f" Page {page} attempt {attempt} failed: {e}, retrying in {wait}s")
await asyncio.sleep(wait)
else:
print(f" Page {page} failed after 3 attempts, skipping")
pages.append("")
await asyncio.sleep(2) # polite delay between pages
return pagesCode Example 2: Parsing Organic and Sponsored Positions
Once you have the HTML, parse it with BeautifulSoup. Iterate all s-search-result divs, read the data-asin, detect Sponsored placements, and compute organic rank:
from bs4 import BeautifulSoup
from dataclasses import dataclass
from typing import Optional
@dataclass
class SearchResult:
asin: str
position: int # raw position on page (1-based)
is_sponsored: bool
organic_position: Optional[int] # organic-only position (None if sponsored)
page_number: int
def parse_amazon_results(html: str, page_number: int) -> list[SearchResult]:
"""Parse an Amazon search results page into structured data."""
soup = BeautifulSoup(html, "html.parser")
results = []
organic_counter = 0
cards = soup.find_all("div", attrs={"data-component-type": "s-search-result"})
for idx, card in enumerate(cards, start=1):
asin = card.get("data-asin", "")
if not asin:
continue
# Detect Sponsored label
sponsored_text = card.get_text(separator=" ", strip=True)
is_sponsored = "Sponsored" in sponsored_text
if not is_sponsored:
organic_counter += 1
organic_pos = organic_counter
else:
organic_pos = None
results.append(SearchResult(
asin=asin,
position=idx,
is_sponsored=is_sponsored,
organic_position=organic_pos,
page_number=page_number,
))
return results
def find_asin_position(results: list[SearchResult], target_asin: str) -> dict:
"""Find the organic and sponsored position of a target ASIN."""
organic_rank = None
sponsored_rank = None
for r in results:
if r.asin == target_asin:
if r.is_sponsored and sponsored_rank is None:
sponsored_rank = r.position
elif not r.is_sponsored and organic_rank is None:
organic_rank = r.organic_position
return {
"asin": target_asin,
"organic_rank": organic_rank,
"sponsored_rank": sponsored_rank,
"page": results[0].page_number if results else None,
}Code Example 3: Full Rank Tracking Pipeline
Combine fetching and parsing into a pipeline that tracks a target ASIN's position across pages 1–5 for a given keyword:
import asyncio
import json
from datetime import datetime, timezone
import csv
async def track_keyword_rank(
keyword: str,
target_asin: str,
max_pages: int = 5,
) -> dict:
"""Fetch and parse Amazon results, return rank data for target ASIN."""
pages_html = await fetch_keyword_pages(keyword, max_pages)
all_results = []
for page_num, html in enumerate(pages_html, start=1):
if not html:
continue
page_results = parse_amazon_results(html, page_num)
all_results.extend(page_results)
rank_info = find_asin_position(all_results, target_asin)
rank_info["keyword"] = keyword
rank_info["timestamp"] = datetime.now(timezone.utc).isoformat()
rank_info["marketplace"] = "amazon.com"
rank_info["total_organic_results"] = sum(1 for r in all_results if not r.is_sponsored)
rank_info["is_indexed"] = rank_info["organic_rank"] is not None or rank_info["sponsored_rank"] is not None
return rank_info
async def batch_track(keywords: list[str], target_asin: str, output_csv: str):
"""Track multiple keywords and append results to CSV."""
with open(output_csv, "a", newline="") as f:
writer = csv.DictWriter(f, fieldnames=[
"timestamp", "keyword", "marketplace", "asin",
"organic_rank", "sponsored_rank", "page",
"total_organic_results", "is_indexed",
])
if f.tell() == 0:
writer.writeheader()
for kw in keywords:
print(f"Tracking: {kw}")
try:
data = await track_keyword_rank(kw, target_asin)
writer.writerow(data)
f.flush()
print(f" Organic rank: {data['organic_rank']}, Sponsored: {data['sponsored_rank']}")
except Exception as e:
print(f" Error: {e}")
await asyncio.sleep(5) # delay between keywords
# Example usage
if __name__ == "__main__":
keywords = ["wireless earbuds", "bluetooth headphones", "noise cancelling earbuds"]
target_asin = "B0CXYZ1234"
asyncio.run(batch_track(keywords, target_asin, "rank_history.csv"))Code Example 4: Playwright for JavaScript-Rendered Pages
Sometimes Amazon serves a JavaScript-heavy version that requires a real browser. Playwright handles this. Here's how to route Playwright through ProxyHat:
from playwright.async_api import async_playwright
import asyncio
PROXY_USER = "user-country-US-session-amz_playwright"
PROXY_PASS = "your_password"
async def fetch_with_playwright(keyword: str, page_num: int, target_asin: str) -> dict:
"""Use Playwright with ProxyHat residential proxy to fetch Amazon results."""
from urllib.parse import quote_plus
url = f"https://www.amazon.com/s?k={quote_plus(keyword)}&page={page_num}"
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=True,
proxy={
"server": f"http://gate.proxyhat.com:8080",
"username": PROXY_USER,
"password": PROXY_PASS,
},
)
context = await browser.new_context(
user_agent=(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36"
),
locale="en-US",
viewport={"width": 1920, "height": 1080},
)
page = await context.new_page()
try:
await page.goto(url, wait_until="domcontentloaded", timeout=30000)
await page.wait_for_selector('[data-component-type="s-search-result"]', timeout=10000)
# Extract results directly in the browser context
results = await page.evaluate("""
() => {
const cards = document.querySelectorAll('[data-component-type="s-search-result"]');
const data = [];
let organicCount = 0;
cards.forEach((card, idx) => {
const asin = card.getAttribute('data-asin');
if (!asin) return;
const text = card.textContent || '';
const isSponsored = text.includes('Sponsored');
if (!isSponsored) organicCount++;
data.push({
asin: asin,
position: idx + 1,
isSponsored: isSponsored,
organicPosition: isSponsored ? null : organicCount,
});
});
return data;
}
""")
# Find target ASIN
organic_rank = None
sponsored_rank = None
for r in results:
if r["asin"] == target_asin:
if r["isSponsored"] and sponsored_rank is None:
sponsored_rank = r["position"]
elif not r["isSponsored"] and organic_rank is None:
organic_rank = r["organicPosition"]
return {
"asin": target_asin,
"keyword": keyword,
"page": page_num,
"organic_rank": organic_rank,
"sponsored_rank": sponsored_rank,
"total_results": len(results),
}
finally:
await browser.close()
# Usage
async def main():
result = await fetch_with_playwright("wireless earbuds", 1, "B0CXYZ1234")
print(result)
asyncio.run(main())Code Example 5: ProxyHat SDK Wrapper with Retry Logic
For production, wrap the proxy logic in a reusable class with exponential backoff, CAPTCHA detection, and circuit breaker patterns:
import asyncio
import logging
from datetime import datetime, timezone
from curl_cffi.requests import AsyncSession
from urllib.parse import quote_plus
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("amazon_tracker")
class ProxyHatAmazonTracker:
def __init__(
self,
username: str,
password: str,
marketplace: str = "amazon.com",
country: str = "US",
):
self.username = username
self.password = password
self.marketplace = marketplace
self.country = country
self.gateway = "gate.proxyhat.com"
self.port = 8080
self.max_retries = 3
self.base_delay = 2
self.failure_count = 0
self.circuit_open = False
self.circuit_threshold = 10 # open circuit after 10 consecutive failures
def _proxy_url(self, session_id: str) -> str:
user = f"{self.username}-country-{self.country}-session-{session_id}"
return f"http://{user}:{self.password}@{self.gateway}:{self.port}"
def _check_captcha(self, html: str) -> bool:
captcha_markers = [
"Type the characters you see in this image",
"api-services-support@amazon.com",
"To discuss automated access",
"captcha",
]
html_lower = html.lower()
return any(marker.lower() in html_lower for marker in captcha_markers)
async def fetch_page(self, keyword: str, page: int, session_id: str) -> str | None:
"""Fetch a single Amazon search page with retry and circuit breaker."""
if self.circuit_open:
logger.warning("Circuit breaker open, skipping request")
return None
url = f"https://{self.marketplace}/s?k={quote_plus(keyword)}&page={page}"
proxy = self._proxy_url(session_id)
for attempt in range(1, self.max_retries + 1):
try:
async with AsyncSession(impersonate="chrome") as s:
resp = await s.get(
url,
proxies={"http": proxy, "https": proxy},
timeout=30,
headers={
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
},
)
if resp.status_code == 200:
if self._check_captcha(resp.text):
logger.warning(f"CAPTCHA detected on page {page}, attempt {attempt}")
self.failure_count += 1
await asyncio.sleep(self.base_delay ** attempt)
continue
self.failure_count = 0
return resp.text
elif resp.status_code == 503:
logger.warning(f"HTTP 503 on page {page}, attempt {attempt}")
self.failure_count += 1
await asyncio.sleep(self.base_delay ** attempt)
else:
logger.error(f"HTTP {resp.status_code} on page {page}")
return None
except Exception as e:
logger.error(f"Request error on page {page}, attempt {attempt}: {e}")
self.failure_count += 1
await asyncio.sleep(self.base_delay ** attempt)
if self.failure_count >= self.circuit_threshold:
self.circuit_open = True
logger.error("Circuit breaker tripped — pausing all requests")
return None
return None
async def track_keyword(
self,
keyword: str,
target_asin: str,
max_pages: int = 5,
) -> dict:
"""Track a keyword's rank for a target ASIN across multiple pages."""
session_id = f"amz_{keyword.replace(' ', '_')[:15]}"
all_results = []
for page in range(1, max_pages + 1):
html = await self.fetch_page(keyword, page, session_id)
if html:
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, "html.parser")
cards = soup.find_all("div", attrs={"data-component-type": "s-search-result"})
organic_count = 0
for idx, card in enumerate(cards, start=1):
asin = card.get("data-asin", "")
if not asin:
continue
is_sponsored = "Sponsored" in card.get_text(separator=" ", strip=True)
if not is_sponsored:
organic_count += 1
all_results.append({
"asin": asin,
"position": idx,
"is_sponsored": is_sponsored,
"organic_position": organic_count if not is_sponsored else None,
"page": page,
})
await asyncio.sleep(2)
# Find target ASIN
organic_rank = None
sponsored_rank = None
for r in all_results:
if r["asin"] == target_asin:
if r["is_sponsored"] and sponsored_rank is None:
sponsored_rank = r["position"]
elif not r["is_sponsored"] and organic_rank is None:
organic_rank = r["organic_position"]
return {
"timestamp": datetime.now(timezone.utc).isoformat(),
"keyword": keyword,
"marketplace": self.marketplace,
"asin": target_asin,
"organic_rank": organic_rank,
"sponsored_rank": sponsored_rank,
"total_results": len(all_results),
"is_indexed": organic_rank is not None or sponsored_rank is not None,
"circuit_open": self.circuit_open,
}
# Usage
async def main():
tracker = ProxyHatAmazonTracker(
username="user",
password="your_password",
marketplace="amazon.com",
country="US",
)
result = await tracker.track_keyword("wireless earbuds", "B0CXYZ1234", max_pages=5)
print(result)
asyncio.run(main())Production Tips for Amazon SERP Scraping
Daily Scheduling
Amazon rankings change throughout the day, but daily snapshots are sufficient for most tracking needs. Use a task scheduler like APScheduler or cron to run your tracker once per day per keyword set. Running at off-peak hours (e.g., 3 AM ET for amazon.com) reduces the chance of hitting rate limits.
Retries with Exponential Backoff
Network hiccups and transient CAPTCHAs are inevitable. Always implement exponential backoff: wait 2 seconds, then 4, then 8. After 3 failures, skip the page and log the error. Don't retry indefinitely — a persistent block means you need a fresh session ID or a different proxy.
CAPTCHA Detection
Amazon's CAPTCHA pages typically return HTTP 503 or contain phrases like "Type the characters you see in this image" or "api-services-support@amazon.com." Detect these in the response body and treat them as a signal to rotate your session. If you see CAPTCHAs on more than 5% of requests, your request rate is too high.
Indexation Checks
Before tracking rank, verify the ASIN is indexed at all. If organic_rank is None across all 5 pages, the ASIN may not be ranking for that keyword. This is normal for new listings or keywords with low relevance. Log is_indexed: false so sellers know the difference between "not ranking" and "ranking on page 6+."
Concurrency
Don't send more than 5–10 concurrent requests per proxy session. Amazon's rate limits are per-IP, so spreading requests across multiple sticky sessions (one per keyword) is safer than hammering a single session. With ProxyHat residential proxies, you can run 50+ keywords in parallel by assigning each a unique session ID.
Ethics and Compliance
Track your own listings and publicly visible competitor data for legitimate market research. Don't scrape at rates that degrade Amazon's service. A good rule of thumb: keep total requests under 100 per hour per marketplace. Consider using Amazon's Selling Partner API (SP-API) for data that's available through official channels — the SP-API's Product Analytics and Search Catalog endpoints can supplement or replace scraping for sellers enrolled in Brand Analytics.
For broader web scraping best practices, see our web scraping use case guide. For SERP tracking methodology, check our SERP tracking guide.
ProxyHat Setup and Configuration
To get started with ProxyHat for Amazon rank tracking:
- Sign up at ProxyHat pricing and choose a residential proxy plan.
- Get your credentials from the ProxyHat dashboard.
- Check available proxy locations to confirm your target countries are supported.
- Review the ProxyHat documentation for advanced configuration options.
The key configuration for Amazon tracking:
- Proxy type: Residential (datacenter IPs are too easily detected by Amazon).
- Country: Match the marketplace (US for amazon.com, DE for amazon.de, GB for amazon.co.uk).
- Session: Sticky sessions per keyword for consistent pagination.
- Rotation: Use a new session ID per keyword batch, not per page.
Key Takeaways
- Amazon's SERP is unique: Ranking is driven by relevance and sales velocity, not just text matching. Track organic position separately from Sponsored placements.
- Parse with data attributes: Use
data-component-type="s-search-result"anddata-asinfor reliable extraction. Detect Sponsored by checking for the "Sponsored" text label.- Residential proxies are essential: Amazon blocks datacenter IPs quickly. Use country-targeted residential proxies (e.g.,
-country-US) with sticky sessions (-session-) for stable pagination.- Production reliability: Implement retries with exponential backoff, CAPTCHA detection, circuit breakers, and indexation checks. Schedule daily runs during off-peak hours.
- Ethics first: Track public data, throttle your requests, and consider Amazon's SP-API for data available through official channels.
FAQ
What is Amazon Keyword Rank Tracking with Proxies?
Amazon keyword rank tracking with proxies is the process of monitoring where a specific ASIN appears in Amazon's organic search results for a given keyword, using residential proxies to access marketplace-specific results from the correct country. Proxies are necessary because Amazon localizes search results by IP geolocation and blocks automated requests from datacenter IPs. A rank tracker fetches search result pages, parses the HTML to extract ASIN positions, separates organic from Sponsored placements, and records the data over time to show ranking trends.
Why does Amazon Keyword Rank Tracking with Proxies matter for proxy users?
Without proxies, Amazon will quickly block your tracking requests — often within 50–100 requests from a single datacenter IP. Residential proxies route requests through real ISP connections, making them appear as genuine shopper traffic. Country-level geo-targeting ensures you see the same results a local shopper would see, which is critical because Amazon's rankings differ by marketplace. Sticky sessions keep the same IP across multiple pages so your pagination is consistent and results aren't jumbled by mid-sequence IP changes.
Which proxy type works best for Amazon Keyword Rank Tracking with Proxies?
Residential proxies are the best choice for Amazon keyword rank tracking. Datacenter proxies are easily detected by Amazon's anti-bot systems and typically get blocked within minutes. Mobile proxies also work well but are more expensive and may not be necessary for rank tracking. Residential proxies offer the best balance of reliability, geo-targeting precision, and cost. With ProxyHat, you can specify the country (e.g., -country-US for amazon.com, -country-DE for amazon.de) and use sticky sessions (-session-) for consistent pagination across result pages.
How do you avoid blocks when implementing Amazon Keyword Rank Tracking with Proxies?
To avoid blocks: use residential proxies with country geo-targeting, assign a unique sticky session per keyword batch, keep requests under 100 per hour per marketplace, add 2–5 second delays between page requests, implement exponential backoff on failures, detect CAPTCHA pages by checking for known markers in the response body, and rotate session IDs between keyword batches. Also use browser-like TLS fingerprinting (via curl_cffi's impersonate parameter or Playwright) and set realistic User-Agent and Accept-Language headers matching the target marketplace.
Can I use Amazon's SP-API instead of scraping for keyword rank tracking?
Amazon's Selling Partner API (SP-API) offers some analytics data through Brand Analytics for enrolled sellers, including search frequency rank and click share for keywords. However, SP-API does not provide real-time organic rank position for a specific ASIN on a specific keyword — that data is only available by parsing the live search results page. For sellers with Brand Analytics access, combining SP-API data with proxy-based rank tracking gives the most complete picture. Always check Amazon's API documentation for current capabilities and rate limits.
