Back to Blog

Grok Bot and Scrapeless: Add a Custom MCP Connector

Isabella Garcia
Isabella Garcia

Web Data Collection Specialist

14-Sep-2026

TL;DR:

  • Scrapeless is not in Grok Bot's Plugins catalog, so it goes in as a custom connector. A Marketplace search for it returns nothing; a Bot registers a custom remote MCP server when you ask it to.
  • One message in English does the setup. Name the server, give the URL https://api.scrapeless.com/mcp, and tell the Bot to authenticate with an x-api-token header.
  • Grok Bot shows an approval card before anything is saved. Check the name, the Remote HTTPS type, the server URL and the header, then save.
  • The header name is x-api-token. A Bearer header is refused with 401 Unauthorized: Missing x-api-token header.
  • Prove it with one tool call, not the status badge. Scrapeless answers the handshake for any key value, and API-key connectors can show a cosmetic "Needs auth" badge while they work.
  • Get a key on the Scrapeless free plan and connect it in a few minutes.

Grok Bot gives every Bot its own cloud computer, a browser and a set of connectors. What it lacks is a direct way to turn a rendered page into clean Markdown, run a crawl in the background or pull Google results from a tool call. The Scrapeless MCP server adds those as tools that every Bot on your account can call.

The connection is a custom remote MCP server, and in Grok Bot you create one by asking a Bot rather than by filling in a settings page. This guide covers the message to send, what to check on the approval card, and how to tell a working connector from one that only looks connected.

Why Scrapeless Is Not in the Plugins Marketplace

Grok Bot shows connectors as Plugins, and that catalog is Cursor's. xAI's Grok Bot guide for teams says there is no separate Grok Bot connector list: connectors appear as plugins in the app, and Grok Bot inherits the team's Cursor connector policy. Scrapeless is not published in that catalog, so searching the Marketplace for it comes back empty.

Anything outside the catalog is added as a custom connector, and a Bot does that for you. You give it the remote server's HTTPS URL and the headers it needs; it registers the server with its Add MCP tool after you confirm. A custom connector is account-wide, so once one Bot adds Scrapeless, the other Bots on the same account can use it too.

What Scrapeless Adds to Your Bots

The server exposes 25 tools, grouped by job:

  • scrape_markdown, scrape_html and scrape_screenshot return a rendered page as Markdown, raw HTML or an image in one call.
  • Sixteen browser_* tools, from browser_create and browser_goto to browser_click, browser_type and browser_snapshot, drive a cloud browser session step by step.
  • crawl_start, crawl_result and crawl_cancel run a crawl in the background and collect it later.
  • google_search and google_trends return search results and trend data, and ai_scraper captures answers from AI assistants such as ChatGPT, Gemini and Perplexity.

A Bot already has a browser of its own. The difference is the output: these tools hand back page content the model can read directly, instead of a session it has to click through.

Prerequisites

  • Grok Bot on an eligible plan. xAI lists SuperGrok Plus, SuperGrok Heavy, Cursor Pro+, Cursor Ultra, and Cursor Teams Standard or Premium. This guide used the Grok Bot desktop app, version 0.47.0, on Windows.
  • A Scrapeless API key from the Scrapeless dashboard.
  • On a Cursor team, a connector policy that allows the server. When the policy blocks one, members see the plugin as Disabled by team admin.

Step 1: Ask a Bot to Add Scrapeless

Open any Bot and send this message, with your key in place of the placeholder:

text Copy
Add a custom remote MCP server.
Name: scrapeless
URL: https://api.scrapeless.com/mcp
Authentication: HTTP header "x-api-token" with the value YOUR_SCRAPELESS_API_KEY
Do not use an Authorization or Bearer header.

The last line does real work. Most connectors authenticate with OAuth or a Bearer token, and a Bot that assumes Bearer builds a header Scrapeless refuses. The endpoint answers such a request with 401 Unauthorized: Missing x-api-token header.

Know where the key ends up when it is part of the message. It stays in that Bot's conversation, and sharing a Bot with your team shows teammates its full history, past conversations included. Grok Bot's help page on secrets recommends keeping API keys out of chat. For a shared Bot, send the same message without the value and type the key into the approval card in Step 2 instead; that is how the key was entered when this setup was tested.

Setting this up now? The Scrapeless free plan covers the connection and your first tool calls.

Step 2: Check the Approval Card and Save

Before the Bot registers anything, Grok Bot shows an Add MCP Server card. Compare each field with what you asked for:

Field Expected value
Name scrapeless
Type Remote HTTPS
Server URL https://api.scrapeless.com/mcp
HTTP headers Key x-api-token, Value: your Scrapeless API key

Leave OAuth Client ID and Client Secret empty, because Scrapeless does not use OAuth. If the header value is blank, type the key there, then choose Save. The card also offers Edit JSON, which holds the same server as a JSON object keyed by its name, one server at a time.

The saved key stays with the connector. xAI's documentation says OAuth tokens stay on Cursor's connector backend and Bots invoke tools without receiving them. Cursor's documentation for its hosted agents adds that header values on HTTP MCP servers are encrypted at rest, redacted after saving, and never placed on the agent's machine; tool calls go through Cursor's backend.

Step 3: Run One Tool Call

Ask the Bot for a small, checkable job:

text Copy
Use the scrapeless MCP server's scrape_markdown tool to fetch https://example.com
and reply with the first heading of the returned page, quoted exactly.

A working connector comes back with the page's heading:

text Copy
"# Example Domain"

A tool call is the test that counts, because the connector's status cannot tell a good key from a bad one. Scrapeless answers the MCP handshake and lists its 25 tools for any key value. The difference shows only when a tool runs: with a wrong key, the result text starts with Failed to fetch data.

The status badge can mislead in the other direction too. Grok Bot's plugin badge counts only an OAuth sign-in as authenticated, so a connector that uses an API key can read Needs auth while its tools work. Cursor staff confirmed the Needs auth badge is cosmetic for connectors like this. Pressing Authenticate does not change it; the tool call above is the check.

Fix the Common Problems

What you see Cause Fix
No Scrapeless result in the Plugins Marketplace It is not a catalog plugin Ask a Bot to add it as a custom connector (Step 1)
Needs auth on the Scrapeless connector The badge counts only OAuth sign-ins Run the tool call from Step 3 instead of Authenticate
401 Unauthorized: Missing x-api-token header The Bot sent Authorization: Bearer Ask it to use the x-api-token header
A tool result starting with Failed to fetch data The key is wrong or revoked Replace the header value with a valid key
Disabled by team admin The team's connector policy blocks the server Ask a Cursor team admin to allow it
Every plugin stops loading at once A connector that accepts connections but never answers Remove the connector that stopped responding

The last row is worth knowing before it happens. Grok Bot discovers all of an account's connectors under one shared time budget, so a single server that never answers its handshake can make every connector come back empty. Scrapeless answered its handshake in under half a second in testing, so it is not the connector that sits silent.

Share Scrapeless Across a Cursor Team

Asking a Bot adds the connector for one account. A team that wants the same connector for every member can publish it as a plugin in a Cursor team marketplace, which Teams and Enterprise plans support. Members then find it in the Plugins catalog under team plugins, and nobody copies a key into a chat.

The plugin's mcp.json holds only a placeholder:

json Copy
{
  "mcpServers": {
    "scrapeless": {
      "url": "https://api.scrapeless.com/mcp",
      "headers": {
        "x-api-token": "${SCRAPELESS_API_KEY}"
      }
    }
  }
}

Its .cursor-plugin/plugin.json declares that placeholder as a variable, so a team admin sets the real key in the Cursor dashboard under PluginsConfigure:

json Copy
{
  "name": "scrapeless",
  "variables": {
    "type": "object",
    "properties": {
      "SCRAPELESS_API_KEY": {
        "type": "string",
        "title": "Scrapeless API key",
        "description": "Sent as the x-api-token header to https://api.scrapeless.com/mcp."
      }
    },
    "required": ["SCRAPELESS_API_KEY"]
  }
}

A Teams plan allows one team marketplace, so a team that already imported one adds the Scrapeless plugin to that repository instead of creating a second.

For more on what the server exposes, read the Scrapeless MCP server announcement. The Cursor setup guide covers the same server in the Cursor editor, the Browser MCP documentation carries the configuration reference, the Scraping API page covers the actors behind the tools, and pricing lists what a call costs.

Conclusion

Adding Scrapeless to Grok Bot takes one message and one approval. The Bot registers https://api.scrapeless.com/mcp as a custom remote MCP server, the approval card confirms the name, type, URL and x-api-token header, and every Bot on the account can then call the 25 tools.

Two checks keep the setup honest. Read the result of a real tool call rather than the status badge, since a wrong key and a working API-key connector can both look unfinished. And decide where the key should live before sending the message: in chat for a private Bot, or in the approval card when teammates can read the conversation.

Ready to give your Bots a live view of the web? Start with the Scrapeless free plan and send the message from Step 1.

FAQ

Q: Can I add a custom MCP server to Grok Bot?

Yes. Ask a Bot to add a custom remote MCP server and give it the server's HTTPS URL and any headers it needs. Grok Bot shows an Add MCP Server approval card with the name, type, URL and headers, and saves the connector when you confirm.

Q: Why doesn't Scrapeless appear when I search Grok Bot's Plugins?

The Plugins catalog lists published plugins, and Scrapeless is not one of them. It connects as a custom connector instead, which does not show up in Marketplace search.

Q: Which header does Scrapeless need in Grok Bot?

x-api-token, with your Scrapeless API key as the value. An Authorization: Bearer header is rejected with 401 Unauthorized: Missing x-api-token header.

Q: Why does my Scrapeless connector say Needs auth?

Grok Bot's badge counts only OAuth sign-ins as authenticated, and Scrapeless uses an API key. Cursor staff describe the badge as cosmetic for connectors like this. Run a tool call to confirm the connector works.

Q: Should I put my API key in the chat message?

It works, but the key then stays in the Bot's conversation, and teammates see that history on a shared Bot. For shared Bots, leave the value out of the message and type the key into the approval card.

Q: Do all my Bots get the Scrapeless tools?

Yes. Custom MCP servers are account-wide, so a connector one Bot adds is available to the other Bots on the same account.

Q: How many tools does Scrapeless add to Grok Bot?

25: three scrape_* tools, sixteen browser_* tools, three crawl_* tools, plus google_search, google_trends and ai_scraper.

At Scrapeless, we only access publicly available data while strictly complying with applicable laws, regulations, and website privacy policies. The content in this blog is for demonstration purposes only and does not involve any illegal or infringing activities. We make no guarantees and disclaim all liability for the use of information from this blog or third-party links. Before engaging in any scraping activities, consult your legal advisor and review the target website's terms of service or obtain the necessary permissions.

Most Popular Articles

Catalogue