How to Benchmark Web Unlockers: A Reproducible Test Design

TL;DR:

• A web unlocker benchmark must measure usable content, not HTTP status alone. A response can be technically successful while containing a challenge page, empty shell, wrong region, or incomplete render.
• The URL sample should be stratified before testing. Separate static pages, server-rendered pages, JavaScript applications, redirects, and traffic-validation challenges so one easy category cannot hide another.
• Latency needs a distribution. Report the median and a high percentile alongside response and content-success rates.
• Effective cost depends on usable deliveries. Divide observed spend by responses that pass the content checks, not by attempts sent.
• Reproducibility comes from a manifest. Record the URL category, expected marker, render requirement, region, request settings, and measurement code for every test cell.

Introduction: A Benchmark Is a Test Contract

A web unlocker benchmark fails when “success” means only that an endpoint returned a response.

The useful question is whether the service delivered the intended public content in a form the downstream parser can use. That requires a test contract: known URLs, category labels, expected content markers, consistent request settings, bounded sample counts, and a scoring rule decided before results are visible.

This guide builds that contract for Scrapeless Web Unlocker. It uses the current unlocker.webunlocker actor and POST /api/v2/unlocker/request surface, but the design is portable to any managed unlocking service.

What a Web Unlocker Does

A web unlocker accepts a target URL and returns the public page content after managing the network and browser work required by that request.

This is different from supplying a proxy address. A proxy changes the network path; the calling application still owns headers, browser execution, cookies, page completeness checks, and response parsing. A managed web unlocker accepts a higher-level request and returns content through an API.

Scrapeless Web Unlocker accepts actor, input, and proxy objects. The current endpoint supports the unlocker.webunlocker actor, a target URL, an HTTP method, redirect control, optional request headers, and a proxy country. The product can return HTML, JSON, Markdown, or a screenshot, and only successful requests are billed.

The benchmark should evaluate the delivered artifact, not infer quality from the infrastructure used to obtain it.

Define Success Before Sending Requests

A benchmark result should pass four independent checks.

1. Transport success. The Scrapeless API request completes with a successful HTTP status under the HTTP semantics standard.
2. Target identity. The response corresponds to the intended URL or an allowed final URL.
3. Content success. A target-specific marker appears and a known block-page marker does not.
4. Completeness. The delivered body contains the required section, record count floor, or rendered element for that test case.

Keep these checks separate in the raw result. A transport success with a missing content marker is not a usable delivery. A valid marker with the wrong locale is also a failure when geography is part of the requirement.

Build a Stratified URL Sample

A reproducible web unlocker benchmark starts with categories that represent the production workload.

Do not draw all URLs from one domain or one difficulty class. If the production workload is e-commerce, news, search, and documentation, keep those groups visible in the results. Each group should contribute the same number of requests or receive an explicit production weight.

The benchmark manifest belongs in version control. A row should include case_id, category, url, expected_marker, blocked_markers, proxy_country, redirect, and render_required. This makes later comparisons use the same targets and scoring logic.

Respect each target's terms and access policies. The Robots Exclusion Protocol defines a standard way site owners communicate crawler preferences, but it does not replace legal review or site-specific authorization.

Send a Controlled Web Unlocker Request

The current Web Unlocker request uses one endpoint and an API key in the x-api-token header.

Note: The benchmark block requires a Scrapeless API key and the Python requests package. Run it in the account whose usage and billing export will be analyzed.

import csv
import os
import statistics
import time
from pathlib import Path

import requests

ENDPOINT = "https://api.scrapeless.com/api/v2/unlocker/request"
API_KEY = os.environ["SCRAPELESS_API_KEY"]
SAMPLES_PER_CASE = 3

CASES = [
    {
        "case_id": "static-control",
        "category": "static",
        "url": "https://example.com",
        "expected_marker": "Example Domain",
        "proxy_country": "ANY",
        "redirect": False,
    },
    {
        "case_id": "html-control",
        "category": "server-rendered",
        "url": "https://httpbin.io/html",
        "expected_marker": "Herman Melville",
        "proxy_country": "ANY",
        "redirect": False,
    },
    {
        "case_id": "js-page",
        "category": "javascript",
        "url": "https://quotes.toscrape.com/js/",
        "expected_marker": "Quotes to Scrape",
        "proxy_country": "ANY",
        "redirect": False,
    },
    {
        "case_id": "redirect-control",
        "category": "redirect",
        "url": "https://httpbin.io/redirect/1",
        "expected_marker": "url",
        "proxy_country": "ANY",
        "redirect": True,
    },
]


def run_case(case):
    payload = {
        "actor": "unlocker.webunlocker",
        "input": {
            "url": case["url"],
            "method": "GET",
            "redirect": case["redirect"],
        },
        "proxy": {"country": case["proxy_country"]},
    }
    started = time.perf_counter()
    response = requests.post(
        ENDPOINT,
        headers={
            "Content-Type": "application/json",
            "x-api-token": API_KEY,
        },
        json=payload,
        timeout=120,
    )
    elapsed_ms = round((time.perf_counter() - started) * 1000, 1)
    response.raise_for_status()
    body = response.text
    return {
        "case_id": case["case_id"],
        "category": case["category"],
        "elapsed_ms": elapsed_ms,
        "api_status": response.status_code,
        "body_bytes": len(response.content),
        "marker_found": case["expected_marker"] in body,
    }


rows = []
for case in CASES:
    for sample in range(1, SAMPLES_PER_CASE + 1):
        row = run_case(case)
        row["sample"] = sample
        rows.append(row)

Path("benchmark-results.csv").write_text("", encoding="utf-8")
with open("benchmark-results.csv", "w", newline="", encoding="utf-8") as handle:
    writer = csv.DictWriter(handle, fieldnames=rows[0].keys())
    writer.writeheader()
    writer.writerows(rows)

latencies = [row["elapsed_ms"] for row in rows]
usable = [row for row in rows if row["marker_found"]]
print({
    "requests": len(rows),
    "usable_deliveries": len(usable),
    "response_rate": len(rows) / len(rows),
    "content_success_rate": len(usable) / len(rows),
    "latency_p50_ms": statistics.median(latencies),
    "result_file": "benchmark-results.csv",
})

The sample is intentionally small and public. Expand it with authorized production targets only after the harness and scoring rules are stable.

Start Scraping with Scrapeless

Power up your web scraping and automation workflow with Scrapeless!
Sign up today and get $5 in free credit — no credit card required.

Claim your free credit now in the Scrapeless Dashboard.

Measure Response Rate and Content Success Separately

Response rate measures whether the API completed successfully. Content-success rate measures whether the returned artifact passed the target-specific checks.

Use these formulas:

• Response rate = successful API responses / total requests.
• Content-success rate = responses passing identity, marker, block-page, and completeness checks / total requests.
• Conditional content quality = usable deliveries / successful API responses.

The third ratio explains whether the service returns technically successful but unusable content. Keep the raw reason for every failed content check, such as missing_marker, known_challenge_marker, wrong_final_url, or below_record_floor.

A fixed text marker is only the starting point. For a product grid, require at least one stable product container and the fields the downstream schema needs. For a JavaScript page, assert a hydrated element rather than a shell heading that exists before rendering.

Report Latency as p50 and p95

Latency should be reported as a distribution because a single average hides the slow edge of the workload.

Record elapsed time from immediately before the API request until the complete response body is available. Report p50 for the typical request and p95 for the slower edge. The W3C Navigation Timing model explains why navigation contains distinct timing phases, but an external API benchmark should use one consistent end-to-end clock unless the provider exposes comparable phase data.

Calculate percentiles for the full sample and per category. A strong static result should not hide a weak JavaScript or regional distribution. Publish the sample size beside every percentile, and avoid comparing percentiles produced by different calculation methods.

Document the quantile method beside the result so another operator can reproduce the same p50 and p95 calculations. The Python statistics documentation makes the method choice explicit and is a useful neutral reference when documenting that calculation.

Calculate Effective Cost per Usable Delivery

Effective cost converts billing into a workload outcome.

Use the invoice or usage export for the measurement window rather than multiplying requests by a price copied from a marketing page. Then calculate:

effective cost per usable delivery = observed spend / usable deliveries

If the production pipeline extracts records, add a second measure:

effective cost per accepted record = observed spend / records passing schema checks

This keeps the benchmark aligned with the business task. A lower request price does not help when the response lacks the fields the pipeline needs. Scrapeless bills only successful Web Unlocker requests, but the benchmark should still apply its own content-quality definition before calling a delivery usable.

Check JavaScript Completeness

JavaScript completeness measures whether the returned content includes the state produced after client execution.

Choose a stable element that appears only after the application renders. Record its selector or text marker in the manifest. A stronger test also checks a minimum record count and one field that is populated from the rendered data rather than the initial HTML shell.

Do not use body size alone. Consent text, navigation chrome, or a challenge document can be large without containing the target data. Pair size with structural assertions.

For screenshot output, define a visual region and an expected element before the run. Screenshots are useful evidence, but they need a human or visual assertion to become a scored result.

Keep the Experiment Reproducible

A reproducible experiment records enough detail for another operator to rerun the same matrix.

Store:

• the manifest and its version;
• the benchmark script and dependency lockfile;
• request settings, proxy country, and redirect policy;
• start and end timestamps in the raw data;
• response status, final URL when available, elapsed time, and body size;
• every content assertion and failure reason;
• the percentile method and aggregation code;
• the usage or billing export used for effective-cost calculations.

Run providers in a balanced order when comparing more than one service. Domain conditions change over time, so completing all requests for one service before starting another can introduce a time bias. Keep concurrency fixed and small enough that the benchmark measures the service rather than a client-side bottleneck.

Read the Results Without Overclaiming

A web unlocker benchmark describes the chosen URLs, settings, location, and measurement window.

Report category-level results before an overall score. Include confidence intervals or raw counts when the sample is small. Separate unsupported cases from failed cases. Note any target removed after the run and preserve the reason, because silently changing the URL set breaks comparability.

Avoid universal claims such as “works on every site.” The defensible conclusion is narrower: which service produced usable content for this manifest under these settings.

Conclusion: Make Usable Content the Unit of Success

A reproducible web unlocker benchmark starts with a manifest and ends with usable deliveries. Stratify the URL set, define content assertions, measure response and content success separately, report p50 and p95 latency, and calculate cost from observed spend per accepted output.

Review Scrapeless pricing, follow the current Web Unlocker documentation, and use the Scraper API guide to place the experiment inside a broader data pipeline.

Ready to Measure Web Unlocker on Your Workload?

Join the Scrapeless community to compare reproducible data-collection experiments: Discord · Telegram.

Create a free account at app.scrapeless.com and run the manifest against a small set of authorized public pages.

FAQ

Q: What should a web unlocker benchmark measure?

A web unlocker benchmark should measure API response rate, content-success rate, latency distribution, JavaScript completeness, and effective cost per usable delivery.

Q: Why is HTTP 200 not enough to count as success?

HTTP 200 only describes the response status. The body may still contain the wrong page, a traffic-validation document, an empty application shell, or incomplete target data.

Q: How many URLs should a benchmark include?

The benchmark should include enough URLs to represent every production category and report uncertainty, but there is no universal minimum. Start with a small balanced manifest, validate the harness, then expand the authorized sample.

Q: How should p95 latency be calculated?

Calculate p95 with one documented quantile method over a clearly defined sample, and publish the request count beside the result. Use the same method for every compared service and category.

Q: Is it legal to benchmark public websites?

Legality depends on jurisdiction, purpose, target terms, and data type. Use authorized public targets, minimize load, respect site policies, and obtain legal advice for the intended production use.

Q: Is Web Unlocker the same as a proxy?

No. A proxy changes the network route, while Web Unlocker accepts a higher-level request and manages the page-access and content-delivery workflow behind an API.