How to Scrape Iframes With Playwright and Scrapeless

TL;DR:

• An iframe contains a separate document that parent-page selectors cannot search. Select the child frame before locating or extracting its elements.
• Playwright exposes every browsing context through page.frames. Identify the intended child by a stable name, URL pattern, or owning iframe element instead of relying on array position.
• Frame and FrameLocator serve different extraction needs. Use Frame for metadata and lifecycle control; use FrameLocator for concise element actions.
• Playwright can automate cross-origin iframe content through the browser protocol. In-page JavaScript remains subject to the browser’s same-origin policy.
• The same iframe extraction logic works locally and through Scrapeless. Moving to Scrapeless Scraping Browser changes browser creation, not frame selection or locator code.
• Free to start. New Scrapeless accounts include free Scraping Browser runtime—sign up at app.scrapeless.com.

Introduction: Start With the Right Document

An iframe is a separate document with its own URL, lifecycle, and DOM. This guide uses Playwright to enumerate the browsing contexts on a real page, identifies the intended child by name and URL, and extracts a form that a parent-page selector cannot see. The same frame logic runs after connecting Playwright to the Scrapeless Scraping Browser.

Why Iframes Need a Different Extraction Path

An <iframe> embeds another browsing context inside the parent page. The element belongs to the parent DOM; the loaded document belongs to a child frame. A selector evaluated against the parent page does not search that child document.

The HTML iframe reference describes that nested context and its own session history. Playwright exposes each context as a Frame, while FrameLocator gives you a concise way to locate elements through the <iframe> element.

Why Use Playwright With Scrapeless

Playwright can inspect frame metadata and interact with a child document even when the iframe is cross-origin. That browser-level access differs from JavaScript running inside the parent page, which is constrained by the same-origin policy.

The Scrapeless Scraping Browser moves Chromium into a hosted anti-detection environment while keeping Playwright's frame APIs. The connection uses connect_over_cdp; every extraction line after browser creation stays standard Playwright.

Prerequisites

• Python 3.10 or later.
• playwright 1.59.0 or a current compatible version.
• Local Chrome/Chromium for the complete executable fallback.
• A funded Scrapeless account and SCRAPELESS_API_KEY for the cloud browser connection. New sessions on the verification account returned code 14500, so the cloud connection is a labelled prerequisite gap.

Install

python -m pip install "playwright==1.59.0"

The runnable example uses the host's /usr/bin/google-chrome. Use a Playwright-managed Chromium if that is how your project provisions browsers.

Connect to the Scrapeless Scraping Browser

Note: This connection needs funded Scraping Browser balance. The final verification account returned insufficient balance, please recharge first. Use your own funded key; the frame enumeration and extraction below ran completely in local Chrome.

import os
from urllib.parse import urlencode
from playwright.sync_api import sync_playwright

params = urlencode({
    "token": os.environ["SCRAPELESS_API_KEY"],
    "sessionTTL": 120,
    "proxyCountry": "US",
})
cdp_url = f"wss://browser.scrapeless.com/api/v2/browser?{params}"

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(cdp_url)
    page = browser.contexts[0].pages[0]
    # Continue with the frame steps below.
    browser.close()

Use the Scrapeless developer docs for current connection parameters.

Step 1 — Open a Page With a Known Child Frame

Selenium maintains a small public test fixture. Its parent page has one iframe named iframe1-name, whose src loads formPage.html:

TARGET_URL = "https://www.selenium.dev/selenium/web/iframes.html"

page.goto(TARGET_URL, wait_until="domcontentloaded")
page.wait_for_selector("#iframe1")
page.wait_for_timeout(500)
print("parent title:", page.title())
print("parent email matches:", page.locator("#email").count())

parent title: This page has iframes
parent email matches: 0

The email input exists, but it belongs to the embedded form document rather than the parent.

Step 2 — Enumerate Frames Before Selecting One

page.frames contains the main frame plus every attached child and descendant:

print("frame count:", len(page.frames))
for index, frame in enumerate(page.frames):
    print(index, repr(frame.name), frame.url, frame.title())

frame count: 2
0 '' https://www.selenium.dev/selenium/web/iframes.html This page has iframes
1 'iframe1-name' https://www.selenium.dev/selenium/web/formPage.html We Leave From Here

Index zero is the main document. Do not turn index one into a permanent contract: another ad, consent widget, or analytics frame can change the order. Name and URL are better evidence.

Step 3 — Select the Frame by Name or URL

The fixture gives you both stable signals:

named_frame = page.frame(name="iframe1-name")
url_frame = next(
    frame for frame in page.frames
    if frame.url.endswith("/formPage.html")
)

assert named_frame is url_frame
print("selected frame:", url_frame.name)
print("selected title:", url_frame.title())

If names are generated, match a bounded URL pattern or find the owning iframe element and call content_frame(). Fail when more than one candidate matches; silently choosing the first duplicate produces plausible data from the wrong widget.

Step 4 — Extract Data From the Child Document

Once you hold the Frame, its locator API behaves like the page's:

email = url_frame.locator("#email")
selected = url_frame.locator("#multi option:checked").evaluate_all(
    "options => options.map(option => option.value)"
)

record = {
    "frame_name": url_frame.name,
    "frame_url": url_frame.url,
    "frame_title": url_frame.title(),
    "email_inputs": email.count(),
    "selected_values": selected,
}

print("email inputs:", record["email_inputs"])
print("selected values:", record["selected_values"])

email inputs: 1
selected values: ['eggs', 'sausages']

The child page has a multiple-select field with those two options selected in its source. This is a stronger check than reading a heading alone: it proves Playwright reached the form document and evaluated state inside it.

Step 5 — Use FrameLocator for Direct Actions

When you do not need frame metadata, FrameLocator is shorter:

frame_ui = page.frame_locator("#iframe1")
print("email through FrameLocator:", frame_ui.locator("#email").count())

The official Playwright frames guide shows the same two levels: locate content through an iframe element, or obtain a Frame object by URL/name.

Ready to run those locators in a hosted browser? Create a free Scrapeless account and replace only browser creation.

Complete Local Verification Script

from playwright.sync_api import sync_playwright

TARGET_URL = "https://www.selenium.dev/selenium/web/iframes.html"

with sync_playwright() as p:
    browser = p.chromium.launch(
        executable_path="/usr/bin/google-chrome",
        headless=True,
    )
    page = browser.new_page()
    page.goto(TARGET_URL, wait_until="domcontentloaded")
    page.wait_for_selector("#iframe1")
    page.wait_for_timeout(500)

    child = next(
        frame for frame in page.frames
        if frame.url.endswith("/formPage.html")
    )
    selected = child.locator("#multi option:checked").evaluate_all(
        "options => options.map(option => option.value)"
    )
    frame_locator_count = page.frame_locator("#iframe1").locator("#email").count()

    assert page.locator("#email").count() == 0
    assert child.name == "iframe1-name"
    assert child.locator("#email").count() == 1
    assert selected == ["eggs", "sausages"]
    assert frame_locator_count == 1

    print("parent title:", page.title())
    print("parent email matches:", page.locator("#email").count())
    print("frame count:", len(page.frames))
    print("child name:", child.name)
    print("child URL:", child.url)
    print("child title:", child.title())
    print("child email matches:", child.locator("#email").count())
    print("selected values:", selected)
    print("FrameLocator email matches:", frame_locator_count)
    browser.close()

The complete run returns two frames, zero parent email matches, one child email match, the expected frame name/title/URL, and ['eggs', 'sausages'] from the child form.

What You Get Back

The verified workflow returns both the extracted form state and enough frame metadata to identify its source:

{
  "parent_email_matches": 0,
  "frame_count": 2,
  "frame_name": "iframe1-name",
  "frame_url": "https://www.selenium.dev/selenium/web/formPage.html",
  "frame_title": "We Leave From Here",
  "child_email_matches": 1,
  "selected_values": ["eggs", "sausages"]
}

The output establishes four facts: the parent document cannot see the email field, the intended child is identifiable by name and URL, the child locator finds one matching input, and the selected form values come from inside that frame. Keep the frame name and URL with each extracted record when a page contains multiple embedded documents.

Cross-Origin Iframes and the Same-Origin Policy

Browser JavaScript such as document.querySelector("iframe").contentDocument can be blocked when the parent and child origins differ. Playwright communicates with the browser automation protocol and exposes the child as a Frame, so it can locate and interact with elements in cross-origin documents.

That does not remove every boundary. Sandbox attributes can restrict scripts, navigation can replace a frame while you are reading it, and browser extensions or PDF viewers may not expose an ordinary HTML document. Inspect the actual frame tree before writing the extraction contract.

Dynamic and Nested Frames

Wait for attachment and navigation

An iframe element can appear before its child has reached the final URL. Wait for a child locator or inspect frame navigation events; the useful readiness marker is the data-bearing element, not merely the parent <iframe> tag.

Expect detachment

Single-page apps sometimes remove and recreate iframes. A saved Frame object can become detached. Resolve the frame again from its stable name, URL, or owner element after the UI transition rather than assuming the old object survives.

Traverse nested frames deliberately

For nested contexts, inspect frame.child_frames or chain FrameLocators. Keep the parent-child path in the output provenance so two identically named widgets cannot be confused.

Conclusion

Iframe scraping starts by choosing the right document. Enumerate the frame tree, select the child by stable name or URL, wait on an element inside it, and keep frame metadata with the extracted record. Use FrameLocator for concise actions and Frame objects when you need identity, navigation, or nested-frame control. Moving the workflow to Scrapeless changes browser creation, not the frame logic.

Ready to Extract Data From Iframes?

Join developers building browser-based extraction workflows in the Scrapeless community: Discord · Telegram.

Start with Scrapeless, review Scrapeless pricing, and read what CDP exposes before moving the same frame workflow to a hosted browser.

FAQ

Q: Why can't the parent page find an element inside an iframe?

The iframe owns a separate document. Parent-page locators search the parent browsing context; select the child Frame or use a FrameLocator first.

Q: Should I select a frame by array index?

No. Frame order changes when pages add widgets or ads. Prefer a stable name, URL pattern, or owning iframe element and fail on ambiguous matches.

Q: What is the difference between Frame and FrameLocator?

A Frame exposes document identity, URL, name, lifecycle, and child frames. FrameLocator provides concise locator operations through a specific iframe element.

Q: Can Playwright access a cross-origin iframe?

Yes, Playwright can automate cross-origin child frames through the browser protocol even when in-page JavaScript cannot read contentDocument because of the same-origin policy.

Q: How do I wait for a dynamically loaded iframe?

Wait for a data-bearing locator inside the frame or a known final frame URL. The iframe element's presence alone does not prove its child document has finished navigating.

Q: What happens if the frame is detached?

The saved Frame no longer represents a live document. Resolve the replacement frame again from stable evidence after the page transition.

Q: Is iframe scraping legal?

Embedding does not change the target's data rights or terms. Review the site and embedded provider's policies, honor robots directives where applicable, keep volume bounded, and seek legal advice for sensitive or commercial collection.