How to Build a Shopify Scraper with Playwright

TL;DR:

• A Shopify scraper should discover product URLs before extracting fields. Same-origin product links are more durable than a theme's styling classes.
• Public structured product data can expose variants without clicking every swatch. Verify the response on the target store before relying on it.
• Playwright can preserve browser context while reading storefront data. Keep product and currency requests inside that context.
• The local example returns variant identifiers and raw price values. Currency conversion and cross-store matching remain separate tasks.
• Free to start. New Scrapeless accounts include free Scraping Browser runtime — sign up at app.scrapeless.com.

Introduction: Product Cards Do Not Describe Every Variant

A storefront product card often summarizes several purchasable variants. Reading the card's displayed price is useful for discovery, but it cannot establish the price and availability of every size or color.

This Shopify product data extraction tutorial uses Playwright to open a public storefront, discover a product link, and read structured variant data in the page's browser context. The example is deliberately bounded to one product. You can inspect the returned fields before building a larger catalog job.

Playwright controls the browser; your extraction contract decides what a valid record means. The price-drop alert workflow explains how recorded observations can become downstream events.

Choose the Extraction Source Before the Selector

The most useful source is the most stable public surface that contains the required fields.

The example uses an observed /products/<handle> link, then reads the product's public .js response. Shopify's Ajax product interface documents this route and locale-aware requests. Its variant array has a documented limit; do not claim full coverage of a large variant catalog merely because the request succeeded.

When a storefront needs interaction to expose an offer, inspect its controls and wait for a product-specific state marker after the interaction. A generic price selector can accidentally pick a recommendation card, a financing amount, or the previous variant's price.

Prerequisites and Installation

The local example needs Node.js, pnpm, and Playwright's Chromium runtime.

Use a Node.js release supported by the installed Playwright package. The verified environment used Node.js 24. The cloud extension additionally requires an active Scrapeless account and a service API key in SCRAPELESS_API_KEY; that credential is a prerequisite, and it is not required for the local collector.

Run these commands in a new working directory:

pnpm add playwright @scrapeless-ai/sdk
pnpm exec playwright install chromium

Playwright's browser library installation separates the package from the browser download. Save JavaScript examples as .mjs files so Node.js treats them as ES modules. The cloud extension uses the Scrapeless Node.js SDK.

Build the Local Discover-and-Extract Collector

The collector finds a checked product anchor and returns each variant from a structured response.

Save this as shopify.mjs and run node shopify.mjs. The specific product is a public example, so recheck the URL and store policy before reusing it. The script reports an absent discovery anchor instead of silently switching to another item.

import { chromium } from 'playwright';

const browser = await chromium.launch({ headless: true });
try {
  const context = await browser.newContext();
  const page = await context.newPage();
  await page.goto('https://www.taylorstitch.com/', {
    waitUntil: 'domcontentloaded', timeout: 45000
  });

  const result = await page.evaluate(async () => {
    const link = Array.from(document.querySelectorAll('a[href]'))
      .map(a => new URL(a.href))
      .find(u => u.origin === location.origin &&
        /^\/products\/jack-in-valley-plaid-waffle-2610\/?$/.test(u.pathname));
    if (!link) throw new Error('Expected product anchor is absent');

    const root = window.Shopify.routes.root;
    const productResponse = await fetch(link.pathname.replace(/\/$/, '') + '.js');
    const cartResponse = await fetch(root + 'cart.js');
    if (!productResponse.ok || !cartResponse.ok) {
      throw new Error('Product or currency response unavailable');
    }
    const product = await productResponse.json();
    const cart = await cartResponse.json();
    if (!product.id || !Array.isArray(product.variants) || !cart.currency) {
      throw new Error('Incomplete public product response');
    }
    return {
      product_id: String(product.id), title: product.title,
      source_url: link.href, currency: cart.currency,
      observed_at: new Date().toISOString(),
      variants: product.variants.map(v => ({
        variant_id: String(v.id), options: v.options,
        price_raw: v.price, compare_at_price_raw: v.compare_at_price ?? null,
        available: v.available, sku: v.sku ?? null
      }))
    };
  });
  console.log(JSON.stringify(result, null, 2));
} finally {
  await browser.close();
}

domcontentloaded is the navigation boundary, rather than a promise that every widget has finished. The checked homepage exposes the product anchor at that boundary. For another theme, wait for the actual link or product control you need before extracting it.

The read-only cart request identifies the presentment currency for that browser context. It does not add an item or start checkout. The product and currency reads still occur at different instants; do not describe them as an atomic inventory snapshot.

Preserve the Returned Variant Contract

The result is a product envelope with a variant array, not a ready-made competitor price history.

The checked product contained six variants and the context reported USD. Those observations establish that this path extracts real product data from the selected source; they do not establish that every Shopify storefront uses the same theme or access configuration.

The product vocabulary is useful for distinguishing identity and descriptive attributes from an offer's changing commercial fields. Preserve that boundary when flattening variants into warehouse rows.

Start Scraping with Scrapeless

Power up your web scraping and automation workflow with Scrapeless!
Sign up today and get $5 in free credit — no credit card required.

Claim your free credit now in the Scrapeless Dashboard.

Extend Discovery Without Losing the Catalog Boundary

Catalog discovery needs an explicit stopping rule and a persistent set of visited URLs.

For a collection-based scraper, inspect the actual next-page link or the storefront's documented pagination mechanism. Keep collection discovery separate from product extraction. Deduplicate canonical product paths so an item appearing in several collections does not become several products.

Record whether a run is a complete approved scope or a partial sample. The single-product example is a sample; it cannot establish the store's catalog size. A variant-array limit, a filtered collection, or an inaccessible page should appear as a coverage limitation in your output.

To scrape dynamic Shopify product pages, preserve the exact option selection and visible state when the structured source is insufficient. Do not copy a parent minimum price into every variant row. Stop when the selected offer cannot be identified confidently.

The Cloud Extension for Playwright

Scrapeless Agent Browser can run a Playwright session remotely while your collector retains its own validation logic.

Use the documented Playwright connection when local browser hosting becomes an operational burden. The SDK creates the browser session; Playwright attaches to its returned endpoint.

Note: The following cloud connection requires an active Scrapeless service API key in SCRAPELESS_API_KEY. The local collection example runs independently. This credential-dependent step must be executed in your account before deploying the cloud path.

import { Scrapeless } from '@scrapeless-ai/sdk';
import { chromium } from 'playwright';

const client = new Scrapeless();
const { browserWSEndpoint } = await client.browser.create({
  sessionName: 'shopify-public-product',
  sessionTTL: 180,
  proxyCountry: 'US'
});
const browser = await chromium.connectOverCDP(browserWSEndpoint);
try {
  const page = await browser.newPage();
  await page.goto('https://www.taylorstitch.com/', {
    waitUntil: 'domcontentloaded'
  });
  console.log(await page.title());
} finally {
  await browser.close();
}

After confirming the connection, apply the same discovery and extraction routine within that session. Country routing is a browser parameter; verify the storefront currency and locale rather than inferring them from the proxy setting alone. Assess Agent Browser and runtime pricing against your collection schedule.

Troubleshoot Missing Data at the Source Boundary

Missing data should be diagnosed before it becomes a stored product record.

Review the store's access rules before scheduling work. The crawler exclusion standard describes robots preferences, while terms and permissions govern the broader access boundary.

Conclusion: Keep Discovery, Context, and Output Together

A Shopify scraper with Playwright becomes easier to maintain when product discovery uses durable links and variant extraction uses a checked structured source. Preserve the browser context, keep money fields explicit, and validate the result before storing it. Expand beyond the sample only after verifying pagination, access limits, and variant completeness for the target storefront.

Ready to Build Your Shopify Collection Workflow?

Join our community to claim a free plan and connect with developers working on browser-based ecommerce collection: Discord · Telegram.

Sign up at app.scrapeless.com and apply the checked extraction contract to your approved product sources.

FAQ

Q: Is scraping public Shopify product data legal?

The answer depends on your jurisdiction, the store's terms, your access method, and intended use. Review those conditions and consult counsel where needed. Keep the workflow on public, permitted product surfaces.

Q: Do you need a proxy to build a Shopify scraper?

The checked local example reads an accessible public source without a proxy. The cloud extension sets proxyCountry to US for consistent routing; currency and market selection still require separate validation.

Q: How should the scraper handle a WAF or Access Denied page?

Treat the result as an unconfirmed collection, not product data. For an authorized cloud workflow, open the homepage first in the same session, then wait for a real product marker. Stop when the storefront does not permit the requested access.

Q: What changes when a Shopify theme rotates its DOM classes?

Reinspect the live markup and preserve the stable product URL or structured source where possible. Update selectors only after confirming that they point to the selected product and variant.

Q: Can this run without an AI agent?

Yes. The local script runs directly with Node.js. An agent can help inspect another storefront, but it is not required to execute the collector.

Q: How should concurrency and product URLs be handled?

Start with one worker per store and the canonical URL discovered from its public links. If the store permits parallel collection, use an initial ceiling of three workers per host or its stricter limit. Keep variant selection separate from canonical product identity.