Using HTTP Proxies in Rust: reqwest, hyper, and Rotating Proxy Pools

Why Rust for Proxy-Aware HTTP Clients?

If you're building high-throughput scraping infrastructure, Rust gives you three things most languages can't: zero-cost abstractions over async I/O, fearless concurrency via tokio, and memory safety without a garbage collector. But configuring a Rust HTTP proxy stack—TLS backends, rotating IPs, auth, and connection pooling—can feel like assembling IKEA furniture with missing screws. This guide puts every piece together.

We'll cover reqwest proxy configuration, lower-level hyper with CONNECT tunneling, concurrent scraping with tokio::task::JoinSet, a trait-based rotating-proxy pool, error handling with thiserror, TLS trade-offs, and compile-time feature flags. Every code block is runnable.

Project Setup

Add these dependencies to your Cargo.toml. We'll use feature flags to keep the build lean—more on that later.

[dependencies]
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "proxy"] }
hyper = { version = "1", features = ["client", "http1"] }
hyper-util = { version = "0.1", features = ["client-legacy", "tokio"] }
hyper-proxy2 = "0.1"
tokio = { version = "1", features = ["full"] }
thiserror = "2"
tracing = "0.1"
tracing-subscriber = "0.3"

reqwest with Proxy Configuration and Authentication

reqwest is the go-to HTTP client for Rust. It supports proxy configuration out of the box—once you enable the proxy feature. Here's a complete example using Rust residential proxies from ProxyHat with geo-targeting and authentication embedded in the proxy URL.

use reqwest::Client;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // ProxyHat residential proxy with US geo-targeting
    let proxy_url = "http://user-country-US:PASSWORD@gate.proxyhat.com:8080";

    let client = Client::builder()
        .proxy(reqwest::Proxy::all(proxy_url)?)
        .user_agent("Mozilla/5.0 (Windows NT 10.0; Win64; x64)")
        .timeout(std::time::Duration::from_secs(30))
        .build()?;

    let resp = client
        .get("https://httpbin.org/ip")
        .send()
        .await?;

    let status = resp.status();
    let body = resp.text().await?;
    println!("Status: {} | Body: {}", status, body);
    Ok(())
}

Custom TLS with rustls

By default, reqwest uses native-tls (OpenSSL/Schannel). Switching to rustls gives you a pure-Rust TLS stack—no system dependencies, consistent cross-compilation, and modern cipher suites. Enable rustls-tls in your features as shown above.

If you need custom root certificates (e.g., corporate MITM proxies), inject them via rustls's RootCertStore:

use reqwest::Client;
use std::fs;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let cert_pem = fs::read("custom-ca.pem")?;
    let cert = reqwest::Certificate::from_pem(&cert_pem)?;

    let proxy_url = "http://user-country-DE:PASSWORD@gate.proxyhat.com:8080";

    let client = Client::builder()
        .proxy(reqwest::Proxy::all(proxy_url)?)
        .add_root_certificate(cert)
        .use_rustls_tls()
        .build()?;

    let resp = client.get("https://httpbin.org/ip").send().await?;
    println!("{}", resp.text().await?);
    Ok(())
}

hyper with HTTP CONNECT for HTTPS Proxies

When you need finer control—custom connection pooling, middleware, or streaming—drop down to hyper. For HTTPS targets through an HTTP proxy, the proxy must establish a CONNECT tunnel. The hyper-proxy2 crate handles this.

use hyper::{Request, Method, Uri};
use hyper_proxy2::{Proxy, Intercept, ProxyConnector};
use hyper_util::client::legacy::Client;
use hyper_util::rt::TokioExecutor;
use tokio::net::TcpStream;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let proxy_uri: Uri = "http://user-country-US:PASSWORD@gate.proxyhat.com:8080"
        .parse()?;
    let proxy = Proxy::new(Intercept::All, proxy_uri);

    let connector = hyper_util::client::legacy::connect::HttpConnector::new();
    let proxy_connector = ProxyConnector::from_proxy(connector, proxy)?;

    let client: Client<_, hyper::body::Incoming> = Client::builder(TokioExecutor::new())
        .build(proxy_connector);

    let req = Request::builder()
        .method(Method::GET)
        .uri("https://httpbin.org/ip")
        .body(hyper::body::Bytes::new())?;

    let resp = client.request(req).await?;
    println!("Status: {}", resp.status());
    let body = hyper::body::to_bytes(resp.into_body()).await?;
    println!("Body: {}", String::from_utf8_lossy(&body));
    Ok(())
}

Intercept::All routes every request through the proxy. Use Intercept::Custom with a closure to selectively proxy based on the destination host—useful for splitting traffic between proxied and direct paths.

Concurrent Scraping with tokio JoinSet

Scraping at scale means parallel requests. tokio::task::JoinSet is purpose-built for bounded concurrent tasks with proper error propagation—unlike join_all, it returns results as they complete.

use reqwest::Client;
use std::sync::Arc;
use tokio::task::JoinSet;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let proxy_url = "http://user-country-US:PASSWORD@gate.proxyhat.com:8080";
    let client = Arc::new(
        Client::builder()
            .proxy(reqwest::Proxy::all(proxy_url)?)
            .pool_max_idle_per_host(0) // disable keep-alive for rotating IPs
            .timeout(std::time::Duration::from_secs(15))
            .build()?
    );

    let targets = vec![
        "https://httpbin.org/ip",
        "https://httpbin.org/headers",
        "https://httpbin.org/user-agent",
        "https://httpbin.org/get",
        "https://httpbin.org/uuid",
    ];

    let mut tasks = JoinSet::new();
    let max_concurrency = 3;
    let sem = Arc::new(tokio::sync::Semaphore::new(max_concurrency));

    for url in targets {
        let client = Arc::clone(&client);
        let sem = Arc::clone(&sem);
        let url = url.to_string();
        tasks.spawn(async move {
            let _permit = sem.acquire().await.unwrap();
            let resp = client.get(&url).send().await?;
            let body = resp.text().await?;
            Ok::<_, reqwest::Error>((url, body))
        });
    }

    while let Some(result) = tasks.join_next().await {
        match result? {
            Ok((url, body)) => println!("✓ {} → {}", url, &body[..body.len().min(80)]),
            Err(e) => eprintln!("✗ error: {}", e),
        }
    }
    Ok(())
}

Production tip: Disable connection pooling (pool_max_idle_per_host(0)) when rotating IPs per request. A reused connection keeps the old proxy IP negotiated during the CONNECT handshake.

A Rotating Proxy Pool Abstraction

Hardcoding a single proxy URL doesn't scale. Let's build a trait-based abstraction that rotates through proxy configurations—country-targeted residential IPs, sticky sessions, and datacenter fallbacks.

use reqwest::{Client, Proxy};
use std::sync::Arc;
use tokio::sync::RwLock;
use std::collections::VecDeque;

/// A single proxy configuration with metadata.
#[derive(Debug, Clone)]
pub struct ProxyConfig {
    pub username: String,
    pub password: String,
    pub host: String,
    pub port: u16,
    pub country: Option<String>,
    pub session_id: Option<String>,
}

impl ProxyConfig {
    pub fn to_url(&self) -> String {
        let mut user = self.username.clone();
        if let Some(ref cc) = self.country {
            user = format!("{}-country-{}", user, cc);
        }
        if let Some(ref sid) = self.session_id {
            user = format!("{}-session-{}", user, sid);
        }
        format!("http://{}:{}@{}:{}", user, self.password, self.host, self.port)
    }
}

/// Trait for proxy rotation strategies.
pub trait ProxyRotation: Send + Sync {
    fn next_proxy(&self) -> Option<ProxyConfig>;
}

/// Round-robin rotation through a VecDeque.
pub struct RoundRobinRotation {
    pool: RwLock<VecDeque<ProxyConfig>>,
}

impl RoundRobinRotation {
    pub fn new(configs: Vec<ProxyConfig>) -> Self {
        Self { pool: RwLock::new(configs.into()) }
    }
}

impl ProxyRotation for RoundRobinRotation {
    fn next_proxy(&self) -> Option<ProxyConfig> {
        let mut pool = self.pool.blocking_write();
        let config = pool.pop_front()?;
        pool.push_back(config.clone());
        Some(config)
    }
}

/// A client that builds a fresh reqwest::Client per request with the next proxy.
pub struct RotatingProxyClient<R: ProxyRotation> {
    rotator: Arc<R>,
    base_builder: reqwest::ClientBuilder,
}

impl<R: ProxyRotation + 'static> RotatingProxyClient<R> {
    pub fn new(rotator: R) -> Self {
        let base_builder = Client::builder()
            .pool_max_idle_per_host(0)
            .timeout(std::time::Duration::from_secs(20));
        Self { rotator: Arc::new(rotator), base_builder }
    }

    pub async fn get(&self, url: &str) -> Result<reqwest::Response, ScrapingError> {
        let config = self.rotator.next_proxy()
            .ok_or(ScrapingError::NoProxyAvailable)?;
        let proxy = Proxy::all(&config.to_url())
            .map_err(|e| ScrapingError::ProxyConfig(e.to_string()))?;
        let client = self.base_builder.clone()
            .proxy(proxy)
            .build()
            .map_err(|e| ScrapingError::ClientBuild(e.to_string()))?;
        client.get(url).send().await
            .map_err(ScrapingError::Request)
    }
}

// Usage:
// let configs = vec![
//     ProxyConfig { username: "user".into(), password: "pass".into(),
//                   host: "gate.proxyhat.com".into(), port: 8080,
//                   country: Some("US".into()), session_id: None },
//     ProxyConfig { username: "user".into(), password: "pass".into(),
//                   host: "gate.proxyhat.com".into(), port: 8080,
//                   country: Some("DE".into()), session_id: Some("sess42".into()) },
// ];
// let rotator = RoundRobinRotation::new(configs);
// let client = RotatingProxyClient::new(rotator);
// let resp = client.get("https://httpbin.org/ip").await?;

Error Handling with thiserror

Scraping is an error-prone business—timeouts, CAPTCHAs, proxy auth failures, TLS handshake errors. A typed error enum with thiserror makes downstream handling clean and logs actionable.

use thiserror::Error;

#[derive(Error, Debug)]
pub enum ScrapingError {
    #[error("no proxy available in the rotation pool")]
    NoProxyAvailable,

    #[error("proxy configuration failed: {0}")]
    ProxyConfig(String),

    #[error("client build failed: {0}")]
    ClientBuild(String),

    #[error("HTTP request failed: {0}")]
    Request(#[from] reqwest::Error),

    #[error("rate limited (HTTP 429) after {attempts} retries")]
    RateLimited { attempts: u32 },

    #[error("CAPTCHA challenge from {host}")]
    Captcha { host: String },

    #[error("response body exceeded {limit} bytes")]
    PayloadTooLarge { limit: usize },
}

/// Retry with exponential backoff on transient errors.
pub async fn fetch_with_retry(
    client: &RotatingProxyClient<RoundRobinRotation>,
    url: &str,
    max_retries: u32,
) -> Result<String, ScrapingError> {
    let mut attempts = 0u32;
    loop {
        attempts += 1;
        match client.get(url).await {
            Ok(resp) => {
                if resp.status() == reqwest::StatusCode::TOO_MANY_REQUESTS {
                    if attempts > max_retries {
                        return Err(ScrapingError::RateLimited { attempts });
                    }
                    let delay = std::time::Duration::from_millis(500 * 2u64.pow(attempts - 1));
                    tokio::time::sleep(delay).await;
                    continue;
                }
                if resp.status().is_success() {
                    return resp.text().await.map_err(ScrapingError::Request);
                }
                return Err(ScrapingError::Request(
                    reqwest::Error::from(resp.error_for_status_ref().unwrap_err())
                ));
            }
            Err(e) if attempts > max_retries => return Err(e),
            Err(_) => {
                let delay = std::time::Duration::from_millis(500 * 2u64.pow(attempts - 1));
                tokio::time::sleep(delay).await;
            }
        }
    }
}

TLS: rustls vs native-tls

Your TLS backend choice affects binary size, cross-compilation, and behavior behind proxies. Here's a practical comparison:

Aspect	rustls	native-tls
Implementation	Pure Rust (ring / aws-lc-rs)	Platform-native (OpenSSL / SChannel / Secure Transport)
Cross-compilation	Excellent—no C toolchain needed	Painful—requires OpenSSL dev headers
Binary size	Smaller (static linking)	Larger or dynamic (.so / .dll)
Cipher suite control	Full control via rustls config	Limited to platform policy
FIPS compliance	Via aws-lc-rs FIPS backend	Via OpenSSL FIPS module
CONNECT tunnel compat	Seamless	Seamless
Corporate MITM CAs	Must add manually	Inherits system trust store

Recommendation: Use rustls-tls for scraping tools you ship as static binaries. Use native-tls only if you need system trust store inheritance or FIPS via OpenSSL.

Compile-Time Feature Flags for Proxy Support

Not every binary needs proxy support. Use Cargo feature flags to conditionally compile proxy logic—keeping your CLI tools lean and your library flexible.

# Cargo.toml
[features]
default = []
proxy-residential = ["reqwest/proxy"]
proxy-socks5 = ["reqwest/socks"]
tls-rustls = ["reqwest/rustls-tls"]
tls-native = ["reqwest/native-tls"]
full = ["proxy-residential", "proxy-socks5", "tls-rustls"]

// src/client/mod.rs
#[cfg(feature = "proxy-residential")]
pub mod proxy;

#[cfg(not(feature = "proxy-residential"))]
pub mod direct;

// src/client/proxy.rs
#[cfg(feature = "proxy-residential")]
pub fn build_client(proxy_url: &str) -> Result<reqwest::Client, reqwest::Error> {
    reqwest::Client::builder()
        .proxy(reqwest::Proxy::all(proxy_url)?)
        .build()
}

// src/client/direct.rs
#[cfg(not(feature = "proxy-residential"))]
pub fn build_client(_proxy_url: &str) -> Result<reqwest::Client, reqwest::Error> {
    reqwest::Client::builder().build()
}

Now cargo build produces a zero-dependency direct client, while cargo build --features full gives you the full residential + SOCKS5 + rustls stack. Your CI matrix can test both paths.

Sticky Sessions for Stateful Scraping

Some targets—login flows, paginated results, cart state—require the same IP across multiple requests. ProxyHat supports sticky sessions via the -session- flag in the username.

use reqwest::Client;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    // All requests with this session ID use the same exit IP
    let session_id = "order-flow-7391";
    let proxy_url = format!(
        "http://user-session-{}-country-US:PASSWORD@gate.proxyhat.com:8080",
        session_id
    );

    let client = Client::builder()
        .proxy(reqwest::Proxy::all(&proxy_url)?)
        .cookie_store(true) // persist cookies across requests
        .build()?;

    // Step 1: Load login page
    let _login_page = client.get("https://example.com/login").send().await?;
    // Step 2: Submit credentials (same IP, same cookies)
    let _result = client
        .post("https://example.com/login")
        .body("user=test&pass=secret")
        .send()
        .await?;

    println!("Sticky session completed on single exit IP");
    Ok(())
}

SOCKS5 Proxies in Rust

For targets that block HTTP proxy headers or when you need UDP relay, SOCKS5 is the answer. Enable the socks feature on reqwest and point to port 1080.

use reqwest::Client;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let proxy_url = "socks5://user-country-JP:PASSWORD@gate.proxyhat.com:1080";

    let client = Client::builder()
        .proxy(reqwest::Proxy::all(proxy_url)?)
        .build()?;

    let resp = client.get("https://httpbin.org/ip").send().await?;
    println!("Exit IP (via SOCKS5): {}", resp.text().await?);
    Ok(())
}

Key Takeaways

• reqwest with the proxy feature is the fastest path to a working Rust HTTP proxy client—configure via Proxy::all() or Proxy::https().
• Use rustls for portable static binaries; native-tls only when you need system trust store or FIPS.
• Disable connection pooling (pool_max_idle_per_host(0)) when rotating IPs per request to avoid stale proxy bindings.
• tokio::task::JoinSet + a semaphore gives you bounded concurrency with proper error collection.
• A trait-based proxy rotation layer decouples your scraping logic from any single provider.
• thiserror enums with retry wrappers turn noisy failures into structured, retryable pipelines.
• Compile-time feature flags keep proxy support opt-in—smaller binaries, faster compile times.
• Sticky sessions via the -session-ID username flag let you maintain state across requests on the same exit IP.

Ready to put this into production? Explore ProxyHat's residential, mobile, and datacenter proxy plans—or dive deeper into web scraping use cases and SERP scraping strategies.