If you're building Kotlin backends or Android apps that fetch data from the web, you'll eventually hit a wall: rate limits, IP bans, or geo-restricted endpoints. Using proxies in Kotlin is the standard way around that wall, but the plumbing differs between HTTP engines. This guide walks through configuring a Kotlin proxy with both Ktor Client and OkHttp, routing through ProxyHat's residential gateway, and hardening the setup for production Kotlin web scraping.
Using Proxies in Kotlin: What You Actually Need
Kotlin runs on the JVM (and Android), so you have two main paths for proxy-aware HTTP: engine-level proxy configuration (where the underlying HTTP engine handles CONNECT tunneling) and system-property proxies (where the JVM's ProxySelector routes everything). The first is what you want for per-client control; the second is a blunt instrument that affects the entire process.
The tricky part is proxy authentication. Unlike browser proxies, programmatic proxies often require credentials passed via the Proxy-Authorization header or the WWW-Authenticate mechanism. Ktor's CIO engine doesn't natively send proxy auth, so you add it manually in defaultRequest. OkHttp handles it via an Authenticator for 407 challenges. SOCKS5 proxies take a different route entirely — you set JVM system properties for credentials.
We'll cover all three scenarios with runnable code.
Project Setup: Ktor 3 and OkHttp Baselines
Gradle Dependencies
For a Kotlin backend project using Ktor 3.x with the OkHttp engine (which gives you fine-grained proxy control and connection pooling), add these to your build.gradle.kts:
dependencies {
implementation("io.ktor:ktor-client-core:3.0.3")
implementation("io.ktor:ktor-client-okhttp:3.0.3")
implementation("io.ktor:ktor-client-content-negotiation:3.0.3")
implementation("io.ktor:ktor-serialization-kotlinx-json:3.0.3")
implementation("com.squareup.okhttp3:okhttp:4.12.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.9.0")
implementation("org.slf4j:slf4j-simple:2.0.16")
}
If you're on Android, you already have OkHttp via the platform or a transitive dependency, and you'd use ktor-client-android or ktor-client-okhttp depending on your needs. The CIO engine is pure-Kotlin and coroutine-native but lacks built-in proxy auth support, which is why we add the header manually.
Raw OkHttp Proxy Baseline
Before touching Ktor, here's the baseline OkHttp approach using java.net.Proxy. This is the most explicit way to route through an HTTP proxy:
import okhttp3.OkHttpClient
import okhttp3.Request
import java.net.InetSocketAddress
import java.net.Proxy
import java.util.concurrent.TimeUnit
fun rawOkHttpProxyExample() {
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("gate.proxyhat.com", 8080))
val client = OkHttpClient.Builder()
.proxy(proxy)
.connectTimeout(10, TimeUnit.SECONDS)
.readTimeout(30, TimeUnit.SECONDS)
.build()
// Proxy-Authorization is required because OkHttp won't prompt for it
// on the initial request — you pre-empt with a Basic header.
val credential = okhttp3.Credentials.basic("user-country-DE-city-berlin", "pass")
val request = Request.Builder()
.url("https://httpbin.org/ip")
.header("Proxy-Authorization", credential)
.build()
client.newCall(request).execute().use { response ->
println(response.body?.string())
// {"origin":"203.0.113.45"} ← the proxy's exit IP
}
}
Note the Proxy-Authorization header. Without it, OkHttp sends the request unauthenticated, the proxy returns 407 Proxy Authentication Required, and then OkHttp's Authenticator kicks in. Pre-emptive auth avoids that round-trip — important when you're doing high-volume Kotlin web scraping and every millisecond counts.
Routing Through ProxyHat with Ktor Client
HTTP Proxy with Geo-Targeting and Sticky Sessions
ProxyHat encodes geo-targeting and session control in the username field, not as query parameters. This means your proxy credentials look like user-country-DE-city-berlin for a Berlin residential exit, or user-country-US-session-abc123 for a sticky US session that keeps the same IP across requests.
Here's a Ktor 3 HttpClient configured with the OkHttp engine, proxy auth in defaultRequest, and JSON content negotiation:
import io.ktor.client.*
import io.ktor.client.engine.okhttp.*
import io.ktor.client.plugins.*
import io.ktor.client.plugins.contentnegotiation.*
import io.ktor.client.request.*
import io.ktor.http.*
import io.ktor.serialization.kotlinx.json.*
import kotlinx.serialization.json.Json
import java.net.InetSocketAddress
import java.net.Proxy
fun createProxyHttpClient(
proxyUser: String = "user-country-DE-city-berlin",
proxyPass: String = "pass"
): HttpClient {
val basic = io.ktor.util.encodeBase64("$proxyUser:$proxyPass".toByteArray())
return HttpClient(OkHttp) {
engine {
config {
proxy(Proxy(Proxy.Type.HTTP, InetSocketAddress("gate.proxyhat.com", 8080)))
}
}
defaultRequest {
header("Proxy-Authorization", "Basic $basic")
header(HttpHeaders.UserAgent, "ProxyHat-Kotlin/1.0")
}
install(ContentNegotiation) {
json(Json { ignoreUnknownKeys = true })
}
// Timeouts matter when routing through residential exits —
// they can be slower than datacenter proxies (200–800ms RTT).
engine {
config {
connectTimeout(10_000, java.util.concurrent.TimeUnit.MILLISECONDS)
readTimeout(30_000, java.util.concurrent.TimeUnit.MILLISECONDS)
}
}
}
}
suspend fun fetchThroughProxy(url: String): String {
val client = createProxyHttpClient()
return try {
client.get(url).bodyAsText()
} finally {
client.close()
}
}
The Proxy-Authorization header is engine-specific. Ktor's CIO engine doesn't have a proxy() config block like OkHttp does — you'd set JVM system properties instead, or use the OkHttp engine as shown. This is the single most common stumbling block for developers new to Ktor client proxy configuration.
Sticky Sessions vs Per-Request Rotation
ProxyHat rotates the exit IP on every request by default. If you need session persistence (e.g., logging into a site and then scraping behind that session), append -session-IDENTIFIER to the username:
| Username pattern | Behavior | Use case |
|---|---|---|
user-country-US | Rotating US residential IPs | Bulk SERP scraping |
user-country-US-session-abc123 | Sticky single IP until session expires (~10 min) | Authenticated scraping, checkout flows |
user-country-DE-city-berlin | City-level geo-targeting, rotating | Local price comparison |
user-country-DE-city-berlin-session-xyz789 | City-level sticky session | Social account management |
Session identifiers are arbitrary strings — use a UUID per logical session. ProxyHat maintains the IP binding for roughly 10 minutes of inactivity, so keep requests flowing if you need long-lived sessions.
SOCKS5 on Port 1080
SOCKS5 is useful when you need TCP-level tunneling (not just HTTP CONNECT) or when the target endpoint uses non-HTTP protocols. ProxyHat exposes SOCKS5 on port 1080. The JVM routes SOCKS credentials via system properties, not via the Proxy-Authorization header:
import java.net.InetSocketAddress
import java.net.Proxy
import okhttp3.OkHttpClient
import okhttp3.Request
import java.util.concurrent.TimeUnit
fun socks5ProxyExample() {
// SOCKS5 credentials go through JVM system properties
System.setProperty("socksProxyHost", "gate.proxyhat.com")
System.setProperty("socksProxyPort", "1080")
System.setProperty("java.net.socks.username", "user-country-US-session-sock1")
System.setProperty("java.net.socks.password", "pass")
val proxy = Proxy(Proxy.Type.SOCKS, InetSocketAddress("gate.proxyhat.com", 1080))
val client = OkHttpClient.Builder()
.proxy(proxy)
.connectTimeout(15, TimeUnit.SECONDS)
.build()
val request = Request.Builder()
.url("https://httpbin.org/ip")
.build()
client.newCall(request).execute().use { response ->
println(response.body?.string())
}
}
The downside of system properties is they're process-global. If you need per-client SOCKS5 credentials (e.g., different sessions for different scrapers in the same JVM), you'll need to isolate with separate class loaders or use a library like sockslib that supports per-connection auth. For most use cases, a single SOCKS5 session per process is fine.
Why Residential Proxies for App and Social Targets
Datacenter IPs are cheap and fast, but they're also obvious. Major platforms — social networks, ticketing sites, sneaker retailers — maintain ASN blocklists that flag traffic from hosting providers like AWS, DigitalOcean, and Hetzner within milliseconds. According to DataDome's documentation on ASN filtering, many anti-bot systems reject requests from datacenter ASNs before even evaluating JavaScript challenges.
Residential proxies route through real ISP-assigned IPs, so the traffic looks like a genuine user on Comcast, Deutsche Telekom, or Vodafone. This is essential for:
- Social media research — Instagram, TikTok, and X aggressively block datacenter ranges.
- App store scraping — Google Play and Apple's App Store rate-limit by IP and ASN.
- SERP tracking — Google serves different results by geography and flags automated datacenter queries with CAPTCHAs.
- Price monitoring — E-commerce sites like Amazon rotate layouts and block datacenter scrapers.
If your Kotlin web scraping target is behind Cloudflare, DataDome, or PerimeterX, residential proxies aren't optional — they're the baseline. See our web scraping use case and SERP tracking guide for domain-specific details.
Coroutine Fan-Out with Rate Control
Here's a production-grade coroutine example that fans out concurrent requests through rotating residential proxies, using a Semaphore to cap concurrency and avoid hammering the target:
import io.ktor.client.*
import io.ktor.client.engine.okhttp.*
import io.ktor.client.plugins.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import kotlinx.coroutines.*
import kotlinx.coroutines.sync.Semaphore
import kotlinx.coroutines.sync.withPermit
import java.net.InetSocketAddress
import java.net.Proxy
import java.util.UUID
data class ScrapeResult(val url: String, val status: Int, val body: String, val durationMs: Long)
suspend fun fanOutScrape(urls: List<String>, maxConcurrency: Int = 20): List<ScrapeResult> = coroutineScope {
val semaphore = Semaphore(maxConcurrency)
val client = HttpClient(OkHttp) {
engine {
config {
proxy(Proxy(Proxy.Type.HTTP, InetSocketAddress("gate.proxyhat.com", 8080)))
// Connection pool tuned for concurrent residential scraping
// OkHttp defaults to 5 idle connections; bump for fan-out.
connectionPool(okhttp3.ConnectionPool(50, 5, java.util.concurrent.TimeUnit.MINUTES))
}
}
defaultRequest {
// Each coroutine gets a unique session for per-request IP rotation.
// For sticky sessions, share a UUID across the batch.
val sessionId = UUID.randomUUID().toString()
val cred = io.ktor.util.encodeBase64("user-country-US-session-$sessionId:pass".toByteArray())
header("Proxy-Authorization", "Basic $cred")
}
}
try {
urls.map { url ->
async(Dispatchers.IO) {
semaphore.withPermit {
val start = System.currentTimeMillis()
try {
val response = client.get(url)
ScrapeResult(url, response.status.value, response.bodyAsText(),
System.currentTimeMillis() - start)
} catch (e: Exception) {
ScrapeResult(url, -1, e.message ?: "error", System.currentTimeMillis() - start)
}
}
}
}.awaitAll()
} finally {
client.close()
}
}
fun main() = runBlocking {
val urls = (1..100).map { "https://httpbin.org/delay/1?i=$it" }
val results = fanOutScrape(urls, maxConcurrency = 20)
val successRate = results.count { it.status == 200 }.toDouble() / results.size * 100
println("Success rate: ${"%.1f".format(successRate)}%")
println("Avg latency: ${results.map { it.durationMs }.average().toInt()}ms")
}
This pattern gives you ~20 concurrent requests through 20 different residential IPs, with a semaphore preventing connection explosion. On a typical residential proxy pool, you can sustain 50–100 concurrent sessions before success rates degrade — tune maxConcurrency based on your target's tolerance.
Production Hardening
OkHttp Authenticator for 407 Challenges
Even with pre-emptive auth, proxies occasionally return 407 (e.g., after a credential rotation). OkHttp's Authenticator interface handles this gracefully:
import okhttp3.Authenticator
import okhttp3.Credentials
import okhttp3.Interceptor
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response
import okhttp3.Route
import java.net.InetSocketAddress
import java.net.Proxy
import java.util.concurrent.TimeUnit
class ProxyAuthenticator(private val user: String, private val pass: String) : Authenticator {
override fun authenticate(route: Route?, response: Response): Request? {
if (response.code != 407) return null
// Avoid infinite loops — OkHttp retries up to 20 times by default
if (response.priorResponse != null) return null
val credential = Credentials.basic(user, pass)
return response.request.newBuilder()
.header("Proxy-Authorization", credential)
.build()
}
}
fun hardenedOkHttpClient(): OkHttpClient {
val proxy = Proxy(Proxy.Type.HTTP, InetSocketAddress("gate.proxyhat.com", 8080))
return OkHttpClient.Builder()
.proxy(proxy)
.authenticator(ProxyAuthenticator("user-country-DE-city-berlin", "pass"))
.connectTimeout(10, TimeUnit.SECONDS)
.readTimeout(30, TimeUnit.SECONDS)
.writeTimeout(15, TimeUnit.SECONDS)
.retryOnConnectionFailure(true)
.build()
}
Retries, Timeouts, and Circuit Breakers
Residential proxies have higher latency variance than datacenter proxies — expect 200–800ms RTT versus 20–50ms for direct connections. Build retry logic with exponential backoff:
import kotlinx.coroutines.delay
import kotlin.math.min
import kotlin.math.pow
import kotlin.random.Random
suspend fun <T> retryWithBackoff(
maxAttempts: Int = 3,
baseDelayMs: Long = 500,
maxDelayMs: Long = 10_000,
block: suspend (attempt: Int) -> T
): T {
var lastError: Exception? = null
repeat(maxAttempts) { attempt ->
try {
return block(attempt)
} catch (e: Exception) {
lastError = e
if (attempt == maxAttempts - 1) throw e
val delayMs = min(maxDelayMs, (baseDelayMs * 2.0.pow(attempt)).toLong())
val jitter = Random.nextLong(0, delayMs / 4)
delay(delayMs + jitter)
}
}
throw lastError ?: IllegalStateException("Unreachable")
}
// Usage:
// val result = retryWithBackoff { attempt -> client.get(url).bodyAsText() }
For circuit-breaker semantics, consider Resilience4j, which integrates cleanly with Kotlin coroutines and provides sliding-window failure rate monitoring.
TLS Configuration
When routing through an HTTP proxy, the proxy performs CONNECT tunneling — your TLS handshake is still end-to-end with the target server. You don't need to disable certificate verification. However, if you're pinning certificates, ensure your ConnectionSpec allows modern TLS:
val specs = listOf(
okhttp3.ConnectionSpec.Builder(okhttp3.ConnectionSpec.MODERN_TLS)
.tlsVersions(okhttp3.TlsVersion.TLS_1_2, okhttp3.TlsVersion.TLS_1_3)
.build()
)
val client = OkHttpClient.Builder()
.connectionSpecs(specs)
.build()
Android NetworkSecurityConfig
On Android, if your targetSdkVersion is 28+ and you're connecting to gate.proxyhat.com over HTTP (the proxy connection itself), add a network_security_config.xml entry to allow cleartext traffic only to the proxy host:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<domain-config cleartextTrafficPermitted="true">
<domain includeSubdomains="false">gate.proxyhat.com</domain>
</domain-config>
</network-security-config>
Reference it in your AndroidManifest.xml: android:networkSecurityConfig="@xml/network_security_config". The actual HTTPS traffic to your target endpoints remains encrypted end-to-end through the CONNECT tunnel.
Ethical and Legal Considerations
Scraping public data is generally legal in the US under the precedent set by hiQ Labs v. LinkedIn, but the Computer Fraud and Abuse Act (CFAA) still applies to unauthorized access. In the EU, GDPR governs personal data — scraping user-generated content that contains personal information may require a lawful basis. Key principles:
- Prefer official APIs. If a platform offers an API, use it. Scraping is slower, more fragile, and more legally exposed.
- Respect
robots.txt. It's not legally binding, but ignoring it signals bad faith. - Rate-limit yourself. Don't send 100 requests/sec to a small site. Use the semaphore pattern above.
- Scrape public data only. Don't bypass authentication or paywalls.
- Honor ToS. Many platforms prohibit scraping in their terms. Violating ToS can breach contract even if the data is public.
For a deeper dive, see our web scraping use case page, which covers compliance patterns.
ProxyHat-Specific Setup
ProxyHat's gateway (gate.proxyhat.com) accepts HTTP on port 8080 and SOCKS5 on port 1080. The ProxyHat documentation mirrors the patterns shown here — the SDK wraps the same Proxy-Authorization header and username-based geo-targeting. For pricing tiers and concurrency limits, see the ProxyHat pricing page. Available locations are listed at /locations.
The SDK itself follows the same architecture: it constructs the Proxy-Authorization header from your credentials, appends geo and session flags to the username, and manages connection pooling. If you're building a Kotlin wrapper, the raw approach above is what you'd implement under the hood.
Key Takeaways
- Engine choice matters. Use Ktor's OkHttp engine for proxy control; CIO requires manual header injection and system-property SOCKS5.
- Auth is in the username. Geo-targeting (
-country-DE-city-berlin) and sticky sessions (-session-abc123) are encoded in the proxy username, not URL parameters.- Pre-empt auth. Add
Proxy-Authorization: Basic ...in Ktor'sdefaultRequestor OkHttp'sAuthenticatorto avoid 407 round-trips.- Residential for anti-bot targets. Datacenter ASNs are blocked in <100ms by Cloudflare, DataDome, and similar systems.
- Cap concurrency. Use
Semaphorewith coroutines — 20–50 concurrent residential sessions is a safe starting point.- Retry with backoff. Residential proxies have higher latency variance; exponential backoff with jitter prevents thundering herds.
- Stay legal. Prefer APIs, respect
robots.txt, and understand CFAA and GDPR implications.
FAQ
What is using proxies in Kotlin?
Using proxies in Kotlin means configuring your HTTP client (Ktor Client, OkHttp, or Java's HttpURLConnection) to route requests through an intermediary server that changes your source IP. In Kotlin, this typically involves setting a java.net.Proxy on the OkHttp engine or adding a Proxy-Authorization header in Ktor's defaultRequest. The proxy handles the CONNECT tunnel, and your TLS handshake remains end-to-end with the target.
Why does using proxies in Kotlin matter for proxy users?
Without a proxy, every request from your Kotlin backend or Android app shares the same IP. Targets rate-limit or ban that IP after a few hundred requests. Proxies let you distribute requests across many IPs, bypass geo-restrictions, and avoid ASN-based blocking. Residential proxies specifically route through ISP-assigned IPs, making your traffic indistinguishable from real users — critical for social media, SERP, and e-commerce scraping.
Which proxy type works best for using proxies in Kotlin?
For anti-bot-protected targets, residential proxies are the best choice because they use real ISP IPs that aren't on ASN blocklists. Datacenter proxies are faster (20–50ms vs 200–800ms) but get flagged by Cloudflare and DataDome. Mobile proxies offer the highest trust score but are more expensive. Use HTTP proxies (port 8080) for most web scraping; SOCKS5 (port 1080) for non-HTTP protocols or TCP tunneling.
How do you avoid blocks when implementing using proxies in Kotlin?
Rotate IPs per request (omit the session flag), use sticky sessions only when login state matters, cap concurrency with a Semaphore (20–50 concurrent is safe), add realistic User-Agent and Accept headers, implement exponential backoff retries, and respect robots.txt. Pre-empt proxy auth with Proxy-Authorization to avoid 407 round-trips. For targets with JS challenges, pair residential proxies with a headless browser.
Can I use the same proxy configuration on Android?
Yes. OkHttp and Ktor Client both work on Android. The main Android-specific consideration is NetworkSecurityConfig — if your targetSdkVersion is 28+, add a cleartext traffic exception for gate.proxyhat.com since the proxy connection itself uses HTTP (the target traffic remains HTTPS end-to-end). Avoid system-property SOCKS5 on Android if your app shares a process with other networking code, as system properties are process-global.
