AI Token Cost: HTML vs Markdown Across GPT, Claude, Gemini

TL;DR:

• Token counts are model-specific measurements. GPT, Claude, and Gemini can split the same bytes differently, so list price alone cannot predict input cost.
• Raw HTML spends context on markup, scripts, styles, and repeated navigation. Main-content Markdown usually preserves more reader-visible information per token.
• Measure format and tokenizer as separate variables. Compare identical pages in raw HTML, extracted text, and Markdown with each provider’s own counter.
• Scrapeless helps control the input before it reaches the model. Render the page, isolate useful content, preserve source URLs, and send only the representation the task needs.

---

An LLM never bills a web page by character count. It bills the tokens produced after a model-specific tokenizer splits the page representation.

That means two engineering choices determine cost: which tokenizer counts the input, and whether the input is raw HTML, plain text, or cleaned Markdown.

What Is an AI Token?

An AI token is a unit produced by a model’s tokenizer and counted against context and usage. Tokens are not universal words or characters. Vocabulary, training corpus, and tokenization algorithm change the boundaries.

The OpenAI tiktoken project exposes the encodings used to measure those boundaries. A count produced for one encoding should not be carried into a different model family as an exact value.

Why the Same Page Produces Different Counts

Common words may fit one vocabulary entry, while uncommon identifiers, code, emoji, and non-English text split into several pieces. Research on tokenization across languages shows that vocabulary choices create unequal representation costs; the language tokenization disparity study measures that effect across language groups.

Structured input adds another source of variation. HTML repeats tag names, attributes, class values, scripts, and style data. JSON and tool schemas add braces, quoted keys, and punctuation. Those bytes are useful to parsers but often irrelevant to the model’s task.

HTML, Text, and Markdown Are Different Inputs

Format	Keeps	Drops	Best fit
Raw HTML	DOM structure, attributes, scripts, styles	Nothing	DOM analysis and selector work
Extracted text	Visible words	Most hierarchy and links	Simple classification or search
Clean Markdown	Headings, lists, tables, links, readable text	Scripts, styles, most layout chrome	Research, summarization, and RAG

Markdown is not automatically the smallest possible representation. It is valuable because it retains useful document structure while removing large classes of browser-oriented bytes.

A Reproducible Token Benchmark

Use a fixed page set that reflects the production workload: documentation, product pages, news, forums, and JavaScript applications if they matter. Save the exact acquired bytes and a hash for each representation.

For every page:

1. Capture raw HTML after the chosen render policy.
2. Extract main-content text.
3. Produce Markdown from the same rendered source.
4. Count all three forms with each provider’s official counter.
5. Record tokens, characters, content hash, extraction settings, and page category.

Do not compare counts gathered on different days from pages that change. Format is the experiment’s independent variable; the source bytes must remain fixed.

Start Scraping with Scrapeless

Power up your web scraping and automation workflow with Scrapeless!
Sign up today and get $5 in free credit — no credit card required.

Claim your free credit now in the Scrapeless Dashboard.

Count With Provider-Native Methods

Use the provider’s supported counter or tokenizer for the model under evaluation. Do not rely on a characters-per-token shortcut for budgeting or context admission.

For long inputs, preserve the counting boundary and document any chunking method. A tokenizer can merge characters across a boundary, so the sum of independently counted pieces may differ slightly from one full-input count.

The cross-tokenizer compression study explains why simple word and character heuristics fail across domains. Treat measured token count as data, not as a universal conversion ratio.

Convert Tokens Into Effective Cost

Effective input cost is measured tokens multiplied by the model’s applicable input price. Keep base input, cached input, long-context tiers, and output usage as separate rows because their billing rules differ.

A fair model comparison uses the same content set and task. If one model receives raw HTML and another receives Markdown, the experiment measures pipeline design and model pricing at the same time.

Where Scrapeless Fits

Scrapeless Universal Scraping API can acquire web content before the model call. The application should keep the source URL and render settings, then choose raw HTML, text, or a cleaned representation based on the downstream task.

Raw HTML remains appropriate when the model must reason about DOM structure. Clean Markdown is usually the better default for question answering and retrieval because headings, lists, links, and tables survive without the full browser document.

The LLM scraper comparison explains how source acquisition and LLM-ready output fit together. Evaluate acquisition volume on the Scrapeless pricing page.

What to Optimize First

Remove content the task does not need before changing models. Navigation, cookie banners, styles, scripts, duplicated mobile markup, and unrelated recommendations consume context without improving most answers.

Then compare tokenizers using the cleaned input. The HTML Living Standard shows why a browser document contains structural and scripting concerns beyond its readable content.

Finally, monitor the production content mix. A benchmark dominated by prose will not predict a workload dominated by code, tables, or multilingual pages.

Conclusion

Token cost begins before the model call. Measure the exact representations your pipeline sends, count them with each model’s own method, and remove browser-oriented bytes that the task does not need. Scrapeless provides the acquisition layer; the application decides which representation becomes model context.

Ready to Measure Web Context Before You Buy It?

Join the Scrapeless community on Discord or Telegram. Start with app.scrapeless.com and benchmark a representative page set in HTML, text, and Markdown.

FAQ

Q: Do GPT, Claude, and Gemini count the same text identically?

No. Each model family uses its own tokenizer and vocabulary, so identical bytes can produce different token counts.

Q: Is one token equal to four characters?

No fixed character ratio is reliable across languages, code, markup, and model families. Use the tokenizer or count method for the exact model.

Q: Does Markdown always use fewer tokens than HTML?

Clean Markdown usually removes scripts, styles, attributes, and repeated navigation, but the result depends on the extractor and page. Measure both representations on the same captured source.

Q: Should raw HTML ever be sent to an LLM?

Raw HTML is justified when the task needs DOM structure, attributes, or selector reasoning. Summarization and research usually need only the main content and links.

Q: How should token cost be compared across models?

Count identical production samples with each model’s supported method, apply the relevant price tier, and keep cached input, long context, and output charges separate.