Ollama Web Scraping: Run a Local LLM Scraper With No API Key

TL;DR:

• Ollama runs the extraction model on your own machine, so the LLM step of a scraper costs no API key and no per-token bill.
• What Ollama does not do is fetch. A local model has no browser and no network reach into a page; it structures text you hand it.
• The gap shows up in one script. A plain GET on a JavaScript-rendered demo page gives you 0 quote blocks; the same URL through the Scrapeless Universal Scraping API with js_render gives you all 10 — and that rendered HTML is what the model reads.
• The extraction is real in this guide. A live run fed three rendered quote blocks to a local qwen2.5:0.5b model and got back clean JSON with the text and author of each — no cloud call anywhere.
• Small models want less noise. Trim the HTML to the content region before the model sees it; a 0.5B model on a focused snippet is fast and accurate, where the full page would not fit.
• Free to start on the fetch side. Create your Scrapeless API key at app.scrapeless.com.

Why run the scraper's model locally

An LLM scraper turns page content into structured records, and that model does not have to live in someone else's cloud. Ollama runs open models on your own hardware behind a small HTTP API, so the extraction step has no API key, no rate limit, and no per-token cost — useful when you are processing many pages or handling data you would rather not send to a third party.

What a local model cannot do is fetch. It has no browser, holds no session, and on a JavaScript-rendered page a plain HTTP request returns markup with none of the content in it. So an Ollama web scraper is two layers: a fetch layer that returns faithful rendered HTML, and the local model as the extraction layer that turns it into records. This guide uses the Scrapeless Universal Scraping API for the first layer. If instead you want to give a local model live web search — the model calling a search tool and reasoning over the results — that is a different job, covered by the local LLM web search guide; this post is about extracting structured data from a specific page.

Prerequisites

• Ollama installed and running, with a small tool-friendly model pulled: ollama pull qwen2.5:0.5b.
• Python 3.9 or later with requests.
• A Scrapeless API key from the dashboard, exported as SCRAPELESS_API_KEY.

Install

Pull a small model and install the one Python dependency. The Ollama server listens on localhost:11434 once it is running.

ollama pull qwen2.5:0.5b
pip install requests

Keep your key in the environment, never in source:

export SCRAPELESS_API_KEY="sk_your_scrapeless_key"

Get a page the model can actually read

Extraction quality is bounded by fetch fidelity, so start there. On a JavaScript-rendered page, the document a plain HTTP client receives is not the document a reader sees — the content arrives only after scripts build the DOM, a lifecycle defined by the HTML scripting specification. One script counts the difference:

# fetch_rendered.py — plain GET vs server-side rendering, same URL
import os

import requests

URL = "https://quotes.toscrape.com/js/"
MARKER = '<div class="quote">'

plain = requests.get(URL, timeout=60).text
print("plain GET chars:", len(plain), "| quote blocks:", plain.count(MARKER))

resp = requests.post(
    "https://api.scrapeless.com/api/v2/unlocker/request",
    headers={"Content-Type": "application/json", "x-api-token": os.environ["SCRAPELESS_API_KEY"]},
    json={"actor": "unlocker.webunlocker", "input": {"url": URL, "js_render": True}},
    timeout=120,
)
resp.raise_for_status()
rendered = resp.json().get("data", "")
print("rendered chars:", len(rendered), "| quote blocks:", rendered.count(MARKER))

The run prints 0 quote blocks for the plain fetch and 10 for the rendered one:

plain GET chars: 5806 | quote blocks: 0
rendered chars: 8940 | quote blocks: 10

Rendering, unlocking, and proxy routing all happen server-side in that one POST — the Universal Scraping API is the fetch layer, and the rendered HTML is what the local model extracts from.

Extract with a local model

Now hand the content to Ollama. Two things keep a small model accurate: trim the HTML to the content region so the model is not reading page chrome, and ask for JSON with format: "json" so the output parses. Ollama's /api/generate endpoint takes the model name and prompt and returns the completion, documented in the Ollama API reference:

# extract_local.py — fetch, trim, and extract with a local Ollama model
import json
import os
import re

import requests

resp = requests.post(
    "https://api.scrapeless.com/api/v2/unlocker/request",
    headers={"Content-Type": "application/json", "x-api-token": os.environ["SCRAPELESS_API_KEY"]},
    json={"actor": "unlocker.webunlocker", "input": {"url": "https://quotes.toscrape.com/js/", "js_render": True}},
    timeout=120,
)
resp.raise_for_status()
html = resp.json().get("data", "")

# Trim to the first three quote blocks — a small local model does better with less noise.
blocks = re.findall(r'<div class="quote">.*?</small>', html, re.S)[:3]
snippet = "\n".join(blocks)

completion = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "qwen2.5:0.5b",
        "prompt": 'Extract every quote from this HTML into JSON with the exact shape '
                  '{"quotes":[{"text":"...","author":"..."}]}. Reply with ONLY the JSON.\n\nHTML:\n' + snippet,
        "stream": False,
        "format": "json",
        "options": {"temperature": 0, "num_predict": 400},
    },
    timeout=600,
)
data = json.loads(completion.json()["response"])

records = data["quotes"]
print("records:", len(records))
print("first author:", records[0]["author"])
print("first text:", records[0]["text"])

The local run returned three records, with the text and author intact on the first:

records: 3
first author: Albert Einstein
first text: The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.

That is the whole scraper, and the model half never touched the network: one POST to Scrapeless to fetch and render, one call to localhost to extract. Scale the number of blocks per call to what your hardware handles — a 0.5B model on CPU is slower than a cloud endpoint, so batch small and keep the input tight.

Get your API key on the free plan: app.scrapeless.com

Advanced patterns

• Match the model to the hardware. qwen2.5:0.5b is fast enough to demonstrate on CPU; a 3B or 7B model extracts more reliably from messier pages if you have the memory and patience. The request shape does not change — only the model string does.
• Trim before you prompt. Isolating the content container is the single biggest quality lever for a small model. Send the repeated block, not the whole document, and token cost and error rate both drop.
• Keep format: "json" and temperature: 0. JSON mode constrains the output to parseable JSON, and zero temperature keeps extraction stable across runs; neither is optional for a scraper.
• Loop in Python, not in the prompt. One content region per request keeps records from bleeding together and makes a bad extraction attributable to a specific page.

Troubleshooting

• Connection refused on localhost:11434. The Ollama server is not running. Start it (ollama serve) and confirm the model is pulled with ollama list.
• Zero blocks to extract. Your fetch returned the pre-render HTML. Count a known element the way the first script does; a near-empty fetch is a rendering problem, fixed with js_render, not a model problem.
• The model returns prose, not JSON. You dropped format: "json". With it, Ollama constrains the output; without it, you are parsing goodwill.
• Extraction is slow or drops records. The model is too small for the input size. Trim to fewer blocks per call, or move to a larger model — small local models trade accuracy for speed, and a shorter, cleaner prompt is how you claw back both.

Conclusion

Ollama earns its place as the extraction layer that runs where you do: open models on your own hardware, no key and no per-token bill, turning page content into JSON. The layer that decides whether any of that is possible is the fetch — the first script's 0-versus-10 count settles it — and one server-rendered POST closes the gap. Wire the two together and a rendered page becomes structured records, with the model half staying entirely on localhost.

Create a free Scrapeless account to get an API key, and the developer docs cover the unlocker.webunlocker parameters. Check Scrapeless pricing when you plan a recurring job.

FAQ

Q: Can Ollama scrape websites by itself?

No. Ollama runs a language model locally; it has no HTTP client for arbitrary pages and renders no JavaScript. It structures text you hand it. Pair it with a fetch layer — here the Scrapeless Universal Scraping API, which renders the page server-side — and the local model handles the extraction.

Q: How is this different from giving a local LLM web search?

Direction. A web-search setup lets the model call a search tool and reason over the results to answer a question; this setup fetches a specific page you choose and has the model extract structured fields from it. One is search-augmented answering, the other is a scraping pipeline — the local LLM web search guide linked above covers the first.

Q: Which local model should I use for extraction?

The smallest one that holds your schema. Extraction with a strict JSON prompt and temperature: 0 is not reasoning-heavy, so a 0.5B model works on a trimmed snippet, as the run above shows. Move to a 3B or larger model only when pages are messy enough that the small one drops fields.

Q: Do I need a GPU?

No. The run in this guide used a 0.5B model on CPU. A GPU makes larger models practical and everything faster, but a small model on CPU is enough to build and test the pipeline.

Q: Is scraping with a local model legal?

Running the model locally does not change the collection rules. Fetch public pages only, respect site terms and the robots directives standardized by the Robots Exclusion Protocol, keep volumes bounded, and handle any personal data under the laws that apply to you.