Skip to content
seamless

How to use proxies with HTTPX

HTTPX takes a proxy through its proxy argument on Client or AsyncClient. Because it supports HTTP/2 and async concurrency, it reaches far higher request rates than Requests, which makes a rotating pool more important rather than less.

Setup

Set up HTTPX in 4 steps

  1. 01

    Install HTTPX

    Install the library, optionally with HTTP/2 support.

    pip install 'httpx[http2]'
  2. 02

    Create a client with a proxy

    Pass the proxy URL when constructing the client so every request through it is routed. Replace PROXY_HOST, PROXY_PORT, USERNAME and PASSWORD with the endpoint and credentials shown in your seamless dashboard.

    import httpx
    
    proxy = "http://USERNAME:PASSWORD@PROXY_HOST:PROXY_PORT"
    
    with httpx.Client(proxy=proxy, timeout=30.0) as client:
        r = client.get("https://api.ipify.org?format=json")
        print(r.json())
  3. 03

    Go async for throughput

    An async client with bounded concurrency is where HTTPX earns its keep. Keep the semaphore modest so you do not overwhelm the target.

    import asyncio, httpx
    
    async def fetch_all(urls):
        limits = httpx.Limits(max_connections=50)
        sem = asyncio.Semaphore(20)
    
        async with httpx.AsyncClient(proxy=proxy, limits=limits, timeout=30.0) as client:
            async def one(url):
                async with sem:
                    return await client.get(url)
            return await asyncio.gather(*(one(u) for u in urls), return_exceptions=True)
  4. 04

    Rotate per worker

    Give each concurrent worker its own session identifier so they do not all share a single exit IP.

    import uuid
    
    def client_for_worker():
        session = uuid.uuid4().hex[:8]
        url = f"http://USERNAME-session-{session}:PASSWORD@PROXY_HOST:PROXY_PORT"
        return httpx.AsyncClient(proxy=url, timeout=30.0)

FAQ

Questions about HTTPX proxies

Anything else? Our team answers on live chat, by email and on Discord.

How do I set a proxy in HTTPX?

Pass proxy='http://username:password@host:port' when creating an httpx.Client or httpx.AsyncClient. Older releases used a proxies argument, which was replaced by the singular proxy in HTTPX 0.26.

Is HTTPX better than Requests for proxy scraping?

For concurrent workloads, yes: async support and HTTP/2 give substantially higher throughput. For simple sequential scripts the difference is negligible and Requests has the wider ecosystem.

Does HTTPX support SOCKS5 proxies?

Yes, with the socks extra installed (pip install 'httpx[socks]'), after which you can pass a socks5:// proxy URL.

Start small. Scale when it works.

Buy a little bandwidth or rent a single ISP proxy for a day, test it on your own targets, then grow from there.