How to Set a User Agent

Scrapeless Agent Browser provides a cloud browser for web automation with configurable browser fingerprints.

Set a user agent by configuring the User-Agent request header in the HTTP client or the supported browser identity settings in a browser environment. Then verify the value received by the server. A configuration value visible in your code is not sufficient evidence that the intended header reached the destination.

The User-Agent field describes the requesting software. It can help a service identify a client or select a representation, but it does not authenticate the client and does not reproduce an entire browser. A useful setup therefore includes both a deliberate value and a clear understanding of what the change can affect.

What Does a User-Agent String Mean?

A User-Agent string is a self-declared description of client software carried in an HTTP request field. The HTTP User-Agent field definition describes product identifiers and optional comments. Servers may use the value for compatibility handling, analytics, or diagnostics.

Browser strings often include historical compatibility tokens. Finding a familiar product name inside the string does not prove that the corresponding engine is running. A client can choose a different string, and several browsers can share parts of the same syntax.

The User-Agent header model also reflects browser efforts to reduce detailed identity information. Avoid designing a collector that depends on every minor operating-system or browser version being exposed in the same format indefinitely.

For an identifiable internal checker, a concise application name and version can be more useful than copying a long browser string. Use an identity that fits the permitted task. Do not claim to be a search-engine crawler or another organization's application to obtain access associated with that identity.

Before You Run the Examples

The examples below require curl or Python with the Requests package, plus network access to the public HTTP header echo endpoint. They send a single GET request and print the value observed by that endpoint. They do not need a Scrapeless account, a proxy, or a browser session.

CatalogAudit/1.0 is an illustrative application identity chosen for this demonstration. Replace it with an appropriate identity for your own application. The expected value is the value you deliberately send, rather than a claim about any browser version.

An echo service is a diagnostic destination. Do not send authentication headers, cookies, or private URLs to it. The examples only submit the demonstration user-agent header, which keeps the check focused on the behavior being taught.

Set a User Agent in curl

curl sets the User-Agent field through the --user-agent option, also available as -A. The curl command-line options define that setting separately from proxy configuration and other request headers.

Run this single request to inspect the value received by the echo endpoint:

curl --silent --show-error --fail --max-time 20 \
  --user-agent 'CatalogAudit/1.0' \
  'https://httpbin.org/user-agent'

The response contains a user-agent field with the value sent by curl. The explicit time limit bounds the command's transfer lifetime, while --fail makes HTTP error responses visible as command failures. This is a small header check, not a test of a complete collection pipeline.

Quote the string so spaces or punctuation remain part of one shell argument when you use a longer identity. Keep credentials out of the string. A user agent is descriptive request metadata and can appear in server logs.

Set a User Agent in Python Requests

Python Requests accepts custom headers through its headers argument. Set the field explicitly on the request and inspect the receiving endpoint's response before applying the same setting to a larger workflow.

This complete example makes one request and validates that the echoed value matches the configured value:

import requests

user_agent = "CatalogAudit/1.0"
response = requests.get(
    "https://httpbin.org/user-agent",
    headers={"User-Agent": user_agent},
    timeout=20,
)
response.raise_for_status()
received = response.json()["user-agent"]
assert received == user_agent
print(received)

The request's timeout bounds network waiting according to the client's timeout behavior; it is not a claim that every possible surrounding business operation finishes within that period. The assertion checks the specific behavior this example teaches: the endpoint received the chosen header value.

For related requests, a Requests Session can hold shared headers and cookies. Decide whether a header belongs to the whole session or just one request. A request-level override is useful for a controlled comparison, but accidental changes inside a session can undermine the interpretation of results.

Verify the Actual Outgoing Request

Verification should compare the configured value with a server observation or a trustworthy capture of the outgoing request. Inspecting only the variable in your script cannot detect a later override, a wrapper that constructs another request, or an intermediary that changes headers.

In a browser, the network panel can show the request headers for an individual request. Select the actual document or API request being investigated, because a page load can involve many separate requests. A value shown for one resource does not establish what every request in the page used.

Keep redirects in mind. The original URL may lead to another destination, and client settings can apply differently across that chain. Record the final URL and inspect the request that produced the content you are evaluating. A correct header on the first request does not prove that the final representation matches your intended test.

If an echo result differs from the configured value, narrow the path. Check the client configuration and any environment-provided proxy settings. Compare the simple request with the application's wrapped request. Change one factor at a time so the observation identifies the cause.

Why Changing User-Agent Does Not Emulate a Device

Changing User-Agent alters a declared identity, while device emulation can involve viewport, input capabilities, browser APIs, and rendering behavior. A desktop networking library that sends a mobile-looking string still does not acquire a mobile browser's execution environment.

The limitations of user-agent detection explain why feature detection is generally more reliable for web application behavior. A site can inspect actual capabilities rather than trusting a product token in a header.

For a mobile-layout test, define the required viewport and interaction conditions as well as the declared identity. For a content-negotiation test, focus on the headers and returned representation. These are different tests and should not be collapsed into a single claim that a request came from a real phone.

TLS behavior also remains separate. A Python HTTP client does not adopt a browser's transport implementation merely because it declares the browser's name. If compatibility depends on the networking stack or page execution, examine that layer directly instead of cycling through unrelated header strings.

How Client Hints Affect Identity Checks

User-agent client hints provide additional structured identity information in browsers that support the mechanism. A site may use those hints alongside the User-Agent string. Changing one field can leave the overall declared environment inconsistent.

Browser tools and automation frameworks differ in which values their identity controls update. Use the documented configuration surface for the actual browser and inspect both network and page-visible properties where relevant. Avoid assuming that a header-only override also updates every JavaScript property.

Keep the test's scope explicit. If the requirement is only to verify an HTTP header, the echo example is enough. If the requirement is to validate a browser identity configuration, inspect the browser environment and the destination's behavior as separate evidence.

Using Agent Browser for Browser-Level Work

Scrapeless Agent Browser is relevant when the task requires a cloud browser rather than only an HTTP transfer. Its custom fingerprint configuration includes User-Agent and other supported identity settings, with documented limits.

Those settings do not imply unrestricted control over every device characteristic. The documented feature does not turn a cloud session into arbitrary physical hardware. Match the supported configuration to the compatibility or collection task and inspect the properties that matter to that task.

The related Python Requests user-agent walkthrough covers the HTTP-client side of the distinction. Use Scrapeless pricing when evaluating browser-based work, since a browser session has a different operating model from a simple local header check.

The examples in this article verify local curl and Requests header configuration. They do not demonstrate an authenticated Agent Browser session or prove how a particular target site will respond to a customized browser identity.

Keep Session Identity Stable and Results Interpretable

A stable identity configuration makes a sequence easier to interpret when its requests belong to one session. Randomly changing the user agent between related operations can alter server-side content selection without representing the user journey the test intended to model.

For a hypothetical compatibility comparison, define separate sessions for the two intended client profiles. Keep the requested resource and other conditions controlled, then compare the actual representation. Record differences in layout or fields instead of assuming a changed header necessarily changes the response.

If the server returns an access restriction, respect that boundary. A user-agent setting is a compatibility and identification control, not an authorization mechanism. Use the service's supported access path for the task.

Conclusion

Set User-Agent through a supported client option and verify it at the receiver. Keep the header's role separate from TLS behavior, browser rendering, and device capabilities. For browser-level tasks, use the documented browser configuration and test the resulting environment directly.