Playwright With Ruby: A Practical Web Scraping Tutorial

TL;DR:

• Ruby can control Playwright through playwright-ruby-client. Install the exact compatible playwright-core version reported by the gem.
• Use locators and explicit waits for dynamic pages. The tutorial extracts two JavaScript-rendered pages and returns structured records.
• Keep provenance in every row. Store the source URL with text and author so pagination errors remain traceable.
• Close browsers and bound pagination. Resource limits and stop conditions matter before a scraper becomes a scheduled job.
• Local Playwright has an operations boundary. Scrapeless Scraping Browser can provide a managed CDP endpoint while Ruby keeps the extraction logic.

Playwright does not publish an official Ruby binding, but the community-maintained playwright-ruby-client provides a practical client for the Playwright protocol. It works well for Ruby applications that need JavaScript rendering, reliable locators, and browser navigation without moving the whole project to Node.js.

This tutorial installs compatible versions, scrapes two pages from a public JavaScript demo, exports structured data, and explains how to move the browser process to Scrapeless when local browser operations become the bottleneck.

Prerequisites

You need Ruby, Bundler, Node.js, and an installed Chromium-based browser. The verified run used Ruby 2.6.10, playwright-ruby-client 1.62.0, playwright-core 1.62.1, and Google Chrome.

The exact Playwright Core version is important. The playwright-ruby-client repository instructs users to query the gem for its compatible protocol version rather than install an arbitrary latest package.

Install Playwright for Ruby

Create a project and add the gem:

# Gemfile
source 'https://rubygems.org'

gem 'playwright-ruby-client'

Install it, print the compatible Playwright version, and install that exact Node package:

bundle install
PLAYWRIGHT_CLI_VERSION=$(bundle exec ruby -e 'require "playwright/version"; puts Playwright::COMPATIBLE_PLAYWRIGHT_VERSION')
npm install "playwright-core@$PLAYWRIGHT_CLI_VERSION"

If no local browser is available, ./node_modules/.bin/playwright-core install chromium downloads the compatible Chromium build. The example below instead points to an existing Chrome executable.

The community client's documentation site covers the supported browser API. The public JavaScript quotes demo is the target used here.

Scrape a Dynamic Page With Ruby

Create scrape_quotes.rb:

require 'json'
require 'playwright'

chrome = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
cli = './node_modules/.bin/playwright-core'
rows = []

Playwright.create(playwright_cli_executable_path: cli) do |playwright|
  playwright.chromium.launch(headless: true, executablePath: chrome) do |browser|
    page = browser.new_page
    page.goto('https://quotes.toscrape.com/js/page/1/', waitUntil: 'domcontentloaded')

    2.times do
      page.locator('.quote').first.wait_for

      page.locator('.quote').all.each do |quote|
        rows << {
          text: quote.locator('.text').text_content,
          author: quote.locator('.author').text_content,
          source_url: page.url
        }
      end

      next_link = page.locator('li.next a')
      break if next_link.count.zero?

      next_link.click
      page.locator('.quote').first.wait_for
    end
  end
end

puts JSON.pretty_generate({ count: rows.length, first: rows.first, last: rows.last })

Run it with:

bundle exec ruby scrape_quotes.rb

The verified run returned 20 records: the first record came from page 1 and the last from page 2. That proves the rendered selector, pagination click, wait, and structured output path on this sample. It is not a throughput benchmark.

Why the Script Uses Locators and Waits

The initial HTML on a dynamic page may not contain the records. page.goto waits for the document event, while locator('.quote').first.wait_for waits for the business element required by the scraper.

That distinction avoids fixed sleeps. A two-second delay can be longer than necessary on one run and too short on another. A locator wait expresses the actual acceptance condition.

Locators also keep queries composable. The script finds each .quote, then scopes .text and .author within that row. This prevents an author from one card being paired with text from another.

Add Safe Pagination and Structured Output

Every pagination loop needs a stop condition. This tutorial uses 2.times, then exits early if no next link exists. A production job can combine a page limit with a visited-URL set and a record deduplication key.

Keep source_url with every record. When a selector changes or a duplicate appears, provenance makes it possible to reproduce the page and distinguish parser errors from source changes.

For file output, replace the final puts with File.write('quotes.json', JSON.pretty_generate(rows)). Write to a temporary file and rename it only after validation if downstream readers may consume the output concurrently.

Control Browser Resources

The block form of launch closes the browser when the block ends, including most exceptions. Scheduled jobs should also limit:

• maximum pages and navigation depth;
• concurrent browser contexts;
• navigation and locator timeouts;
• screenshot and trace retention;
• accepted response types and maximum payload size.

A page that returns status 200 may still be the wrong representation. Validate the final URL, page title, required selector, and at least one business identifier before saving records. The HTTP semantics specification explains response status, but application-level identity remains the scraper's responsibility.

Where Local Playwright Stops

Local Playwright keeps code simple during development. In production, the team also maintains compatible browser binaries, OS dependencies, process cleanup, memory allocation, geographic routing, session isolation, and diagnostics.

The Ruby client requires a Playwright server or CLI that speaks the compatible protocol. Upgrading the gem without its matching playwright-core version can break that boundary. Pin both packages and update them together.

Scrapeless Scraping Browser moves the browser process to a managed service. Scrapeless documents both its SDK and a CDP connection URL in the Playwright integration guide.

Connect Ruby to a Scrapeless Browser Session

Prerequisite: a live cloud connection requires a reader-owned Scrapeless API key. The local Ruby scraping script was run successfully; the cloud handshake was not executed in this environment because no SCRAPELESS_API_KEY was available.

The documented architecture has two sides:

1. create a managed Scrapeless browser session with the Scrapeless SDK or documented browser endpoint;
2. give Ruby the resulting WebSocket endpoint and connect through the client-supported remote browser method.

The Ruby client documents Playwright.connect_to_browser_server for a Playwright server WebSocket. Scrapeless exposes a CDP endpoint, so confirm the current Ruby client's CDP compatibility before wiring it directly. When direct compatibility is uncertain, keep a small Node connection service at the browser boundary and send structured page results to Ruby. Do not silently substitute one WebSocket protocol for another.

That explicit boundary is safer than publishing an untested connection snippet. It preserves the verified Ruby extraction logic while leaving session creation and protocol adaptation in a component that can be integration-tested with a real account.

Conclusion

Playwright with Ruby is practical when the gem and playwright-core versions are paired exactly. Locators, business-element waits, bounded pagination, provenance, and browser cleanup turn a demo into a reliable starting point.

Use local Chrome while developing. Move the browser process to Scrapeless when installation, scaling, routing, and diagnostics become a recurring operational burden, and test the remote protocol boundary with a real credential before deployment.

Build the Browser Boundary Once

Read the Playwright Stealth guide, compare current pricing, then create a Scrapeless account and verify a managed session with your own API key.

FAQ

Q: Does Playwright officially support Ruby?

Microsoft Playwright does not publish an official Ruby binding; playwright-ruby-client is a community-maintained client.

Q: Which Playwright version should Ruby use?

Query Playwright::COMPATIBLE_PLAYWRIGHT_VERSION from the installed gem and install that exact playwright-core version.

Q: How do I wait for JavaScript content in Ruby Playwright?

Wait for a locator that represents the required business content instead of relying on a fixed sleep.

Q: How should pagination be bounded?

Use a maximum page count, stop when the next link disappears, and track visited URLs or record keys.

Q: Can Ruby connect directly to Scrapeless Scraping Browser?

Scrapeless exposes a CDP endpoint, while the Ruby community client documents a Playwright-server connection; verify protocol compatibility with a real account or use a small tested Node boundary.