How to Build a Rotating Proxy Script in Python 2026
Build a rotating proxy script in Python and Node.js with copy-paste code: list rotation, retries, health checks, provider gateways, and the mistakes to avoid.
Send enough requests from a single IP address and you will hit a wall fast — many websites start throttling or blocking after just a few dozen requests a minute. With automated traffic now making up nearly half of all web activity according to Imperva, sites are more aggressive than ever at spotting and banning repeat visitors.
The fix is proxy rotation: spreading your requests across many IP addresses so no single one looks suspicious. In this guide you will build a rotating proxy script from scratch in both Python and Node.js, complete with retries, health checks, and provider gateways.
By the end you will understand exactly how rotation works, have copy-paste code for both languages, and know the mistakes that silently get scrapers blocked. Let us build something that actually survives in production.
What Is a Rotating Proxy?
A rotating proxy is a setup that changes your outgoing IP address automatically across requests. Instead of every request leaving from one address, each one (or each session) uses a different IP drawn from a pool, so the target site sees traffic that looks like many separate visitors.
This matters because rate limits and bans are almost always tied to an IP. A single proxy just moves the bottleneck — it still gets blocked once it sends too much. Rotation distributes the load, dramatically lowering the chance any individual IP trips a limit.
If you are new to proxies generally, our proxy provider directory breaks down the types, and our Selenium scraping guide shows where rotation fits into a real scraper.
Two Ways to Rotate Proxies
Before writing code, understand the two fundamental approaches. Your script will look different depending on which you choose.
| Approach | How it works | Best for |
|---|---|---|
| Client-side list rotation | You hold a list of proxies and pick one per request in your code | Small pools, full control, cheaper datacenter proxies |
| Provider rotating endpoint | One gateway URL that rotates the IP for you automatically | Large-scale jobs, residential pools, zero list management |
Client-side rotation teaches you how everything works and is perfect for smaller pools. Provider gateways are what you reach for at scale, because the provider handles the pool of millions of IPs for you. We will build both.
What You Will Need
- Python 3.8+ with the
requestslibrary, or Node.js 18+ withaxios. - A list of proxies or a provider gateway, in the format
user:pass@ip:port. - A test endpoint like
httpbin.org/ip, which simply returns the IP it sees — perfect for confirming rotation works.
Building a Rotating Proxy Script in Python
Python is the most popular language for this, so we will start here and go from a basic rotator to a production-ready function.
Step 1 — A simple proxy list rotator
The simplest rotation just picks a random proxy from a list for each request. Point it at httpbin and the returned IP changes on every run.
import random
import requests
# Pool of proxies in user:pass@ip:port format
PROXIES = [
"http://user:pass@198.51.100.1:8000",
"http://user:pass@198.51.100.2:8000",
"http://user:pass@198.51.100.3:8000",
]
def get_random_proxy():
proxy = random.choice(PROXIES)
return {"http": proxy, "https": proxy}
resp = requests.get("https://httpbin.org/ip", proxies=get_random_proxy(), timeout=10)
print(resp.json()) # the exit IP changes each runStep 2 — Add retries and rotation on failure
Real proxies fail. A production script must catch errors, rotate to a fresh proxy, and retry — never crash on the first dead IP.
import random
import requests
from requests.exceptions import RequestException
def fetch(url, max_retries=3):
for attempt in range(max_retries):
proxy = random.choice(PROXIES)
try:
resp = requests.get(
url,
proxies={"http": proxy, "https": proxy},
timeout=10,
headers={"User-Agent": "Mozilla/5.0"},
)
resp.raise_for_status()
return resp
except RequestException:
continue # rotate to a different proxy and try again
raise RuntimeError(f"All {max_retries} attempts failed for {url}")
print(fetch("https://httpbin.org/ip").json())Step 3 — Rotate with a provider gateway
At scale, let the provider do the work. A single rotating endpoint hands you a new IP on every request, so there is no list to manage or maintain.
import requests
# One endpoint that rotates the IP automatically on every request
GATEWAY = "http://USER:PASS@gate.provider.com:7000"
proxies = {"http": GATEWAY, "https": GATEWAY}
for _ in range(3):
resp = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
print(resp.json()) # a different IP each timeBuilding a Rotating Proxy Script in Node.js
Prefer JavaScript? The same patterns translate cleanly to Node.js using axios and a proxy agent.
Step 1 — Rotate a proxy list with axios
Install the dependencies, then pick a random proxy per request with an HTTPS proxy agent.
// npm i axios https-proxy-agent
import axios from "axios";
import { HttpsProxyAgent } from "https-proxy-agent";
const PROXIES = [
"http://user:pass@198.51.100.1:8000",
"http://user:pass@198.51.100.2:8000",
"http://user:pass@198.51.100.3:8000",
];
const randomProxy = () => PROXIES[Math.floor(Math.random() * PROXIES.length)];
const res = await axios.get("https://httpbin.org/ip", {
httpsAgent: new HttpsProxyAgent(randomProxy()),
timeout: 10000,
});
console.log(res.data); // exit IP rotates each runStep 2 — Add retry and rotation logic
Wrap the request so a failed proxy triggers a rotation and a retry instead of throwing immediately.
import axios from "axios";
import { HttpsProxyAgent } from "https-proxy-agent";
const randomProxy = () => PROXIES[Math.floor(Math.random() * PROXIES.length)];
async function fetchWithRotation(url, maxRetries = 3) {
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
return await axios.get(url, {
httpsAgent: new HttpsProxyAgent(randomProxy()),
timeout: 10000,
headers: { "User-Agent": "Mozilla/5.0" },
});
} catch (err) {
if (attempt === maxRetries - 1) throw err;
// otherwise rotate to a new proxy and retry
}
}
}
const { data } = await fetchWithRotation("https://httpbin.org/ip");
console.log(data);Validating and Managing Your Proxy Pool
If you rotate a list yourself, dead proxies are inevitable. A quick health check keeps only working IPs in rotation, which is one of the biggest reliability upgrades you can make.
import requests
def is_alive(proxy, test_url="https://httpbin.org/ip", timeout=8):
try:
r = requests.get(test_url, proxies={"http": proxy, "https": proxy}, timeout=timeout)
return r.status_code == 200
except requests.RequestException:
return False
live = [p for p in PROXIES if is_alive(p)]
print(f"{len(live)} of {len(PROXIES)} proxies are alive")Run this periodically and rotate only across the live list. With a provider gateway you can skip this entirely, since the provider retires bad IPs for you.
Best Rotating Proxy Providers to Plug In
Your script is only as good as the IPs behind it. Free proxy lists are slow and short-lived; for anything serious, a paid rotating network is worth every penny. These are the providers we rate most highly.
Decodo
Decodo pairs a large residential pool with a rotating gateway that drops straight into the Step 3 pattern above, plus a friendly dashboard for managing endpoints and sticky sessions.
Its balance of price, ease of use, and reliability makes it a great default for developers who want rotation that just works without fiddly list management.
Oxylabs
Oxylabs is the enterprise option, with a massive pool, excellent geo-targeting, and rotating endpoints built for heavy, sustained scraping workloads.
If you are running large jobs where uptime and success rate matter more than cost, its reliability and support justify the premium price.
IPRoyal
IPRoyal is the value pick, well known for non-expiring traffic that suits intermittent rotation jobs where you do not want unused bandwidth to vanish each month.
It offers both rotating and sticky residential sessions at approachable prices, making it ideal for individuals and smaller projects.
Webshare
Webshare is the developer favorite for affordable, list-style proxies, with a free tier that is perfect for testing the Step 1 and Step 2 scripts above.
Its clean API lets you pull a fresh proxy list programmatically, which pairs naturally with the client-side rotation approach.
Rotating vs Sticky Sessions: Which Do You Need?
Not every task wants a new IP on every request. There are two modes, and choosing the right one prevents a lot of mysterious failures.
| Mode | Behavior | Use it for |
|---|---|---|
| Rotating | A new IP on almost every request | Scraping many independent pages, spreading load |
| Sticky session | The same IP held for a set time, often 1 to 30 minutes | Logins, carts, and multi-step flows that must stay on one IP |
If you rotate the IP mid-checkout or mid-login, the site sees your session jump across the world between clicks and flags it instantly. Use rotation for breadth and sticky sessions for continuity. Most providers let you choose per request by tweaking the gateway username or port, so you can mix both in a single script.
How Many Proxies Do You Actually Need?
This is a frequent question with no single answer, but a useful rule of thumb applies: the more requests per minute you send and the stricter the target, the larger the pool you need. A few dozen datacenter IPs can handle a light scraping job, while aggressive, large-scale scraping of a protected site can demand thousands of rotating residential IPs.
Rather than guessing, start small, watch your block rate, and scale the pool until blocks fall to an acceptable level. With a provider gateway this is effortless, because you tap into the provider entire network and only pay for the bandwidth you actually use, instead of buying and babysitting a fixed list.
Plugging Rotation Into a Real Scraper
The functions above are the building blocks; in practice you wire them into a larger scraper. The principle stays the same across every tool: resolve a proxy, attach it to the request, and rotate on failure.
In a requests-based crawler, call your fetch function for every URL and let the retry loop handle rotation transparently. In Scrapy, set the proxy through the request meta or a downloader middleware so every spider request rotates automatically. For browser-based scraping with Selenium or Playwright, the proxy is passed at launch — our Selenium scraping guide walks through that exact setup, and the same idea applies when you drive a headless browser for browser automation.
The takeaway is that rotation logic belongs in one small, reusable layer. Build it once, test it against httpbin, and drop it into whatever scraping stack you happen to use.
Common Mistakes to Avoid When Rotating Proxies
Most rotation failures come from the same avoidable errors. Sidestep these and your success rate climbs sharply.
Rotating only the IP, not the headers
A fresh IP paired with the same User-Agent and headers on every request is still a pattern. Vary your User-Agent and headers alongside the IP so each request looks like a genuinely different client.
Hammering without delays
Rotation is not a license to flood a site. Firing hundreds of requests per second from a rotating pool still looks like an attack. Add small, randomized delays between requests to stay under the radar.
Ignoring failed proxies
Retrying the same dead proxy wastes time and inflates failure rates. Track which proxies fail, drop them from rotation, and run periodic health checks so your pool stays clean.
Using free proxies for serious work
Free proxy lists are slow, overcrowded, often already banned, and sometimes outright malicious. For anything that matters, use a reputable paid provider — the reliability difference is enormous.
Mismatching proxy type to the task
Datacenter proxies are fast and cheap but easy to detect; residential and mobile proxies look like real users but cost more. Using the wrong type for a sensitive target is a common and costly mistake.
Best Practices for Rotating Proxies
- Rotate headers with IPs — vary the User-Agent so each request looks like a different client.
- Always retry on failure — catch errors, switch proxies, and try again before giving up.
- Pace your requests — add randomized delays to mimic human timing and respect the target.
- Health-check your pool — drop dead proxies automatically, or let a provider gateway handle it.
- Match the proxy type — residential for sensitive sites, datacenter for speed and volume. Compare options in our proxy directory.
Frequently Asked Questions
The Bottom Line
Building a rotating proxy script is one of the most useful skills for anyone scraping or automating the web. The core pattern is simple — pick a proxy, send the request, retry on failure with a fresh IP — and the same logic works whether you write it in Python or Node.js.
Start with client-side list rotation to learn the mechanics, add retries and health checks for reliability, then graduate to a provider gateway when you scale. Pair your script with a quality network from our proxy provider directory, rotate your headers, pace your requests, and your scrapers will keep running long after a single-IP setup would have been banned.
Keep Reading
More articles you might enjoy