How to benchmark a proxy provider without being misled

Provider-published success rates are measured against targets the provider chose, at a time it chose, with a definition of success it wrote. They are not wrong, but they are not about your workload. The only number that predicts your results is one you measured on your own targets.

Define success before you start

A 200 status is not success. Pages that return 200 with a challenge, a consent wall or an empty shell are failures for any real purpose. Decide a check per target: a string that must be present, a selector that must match, a minimum body size. Apply it identically to every provider you test.

Sample enough, and spread it out

Run each provider on the same targets, interleaved, not one after the other. Defences change by the hour, and sequential runs compare different hours.
Use at least a few hundred requests per target per provider before reading a percentage. A success rate from thirty requests has an interval too wide to rank anything.
Repeat across different times of day and at least two days. A single good afternoon is an anecdote.

Report distributions, not averages

Time to first byte has a long tail. The mean hides it, and the tail is what stalls a pipeline. Report the median and the 95th percentile. Report success rate per target, because an average across ten easy targets and one hard one tells you nothing about the hard one.

Measure cost per success, not unit price

A cheaper unit price with a lower success rate can cost more per good page. The comparison that matters is price divided by success rate, plus bytes wasted on failed attempts for traffic-priced products. The cost tool on this site does the arithmetic.

A starting harness

bench.py

import statistics
import requests

PROXY = "http://USERNAME:PASSWORD@GATEWAY_HOST:PORT"
URL = "https://example.com/"
MUST_CONTAIN = "Example Domain"
N = 300

ok, headers_at = 0, []
for _ in range(N):
    try:
        r = requests.get(URL, proxies={"http": PROXY, "https": PROXY}, timeout=30)
        headers_at.append(r.elapsed.total_seconds())
        ok += r.status_code == 200 and MUST_CONTAIN in r.text
    except requests.RequestException:
        pass

headers_at.sort()
print(f"success {ok}/{N}")
if headers_at:
    print(f"response p50 {statistics.median(headers_at):.2f}s  p95 {headers_at[int(len(headers_at) * 0.95) - 1]:.2f}s")

The harness is deliberately small. Replace the target and the success check, and run it against every provider under test, including us.