How Does a CDN Work?

Scrapeless Proxies let applications choose supported exit locations for regional web requests.

A CDN, or content delivery network, is a distributed set of servers that delivers content through network locations closer or better connected to users. A CDN commonly caches eligible responses and forwards other requests to an origin service. It can reduce repeated origin work, but its behavior depends on routing and cache configuration.

For a data collector, the practical consequence is that the same URL does not necessarily mean a fresh origin response. A request may receive a stored representation, a newly fetched representation, or a response that varies with region and request headers. Understanding those paths helps explain why observations differ.

What Are the Origin and the Edge?

The origin is the source service behind the CDN, while an edge location handles requests within the CDN's delivery network. The content delivery network model distributes delivery across locations instead of requiring every client to communicate with one origin location.

The edge may serve a response from storage, forward a request, terminate TLS, or apply configured logic. Not every CDN performs every function for every request. A useful explanation therefore separates the network's capabilities from the policy enabled on a particular hostname.

The origin can be a web application, an object store, or another service endpoint. It may itself use load balancing and multiple servers. CDN and origin are logical roles in the delivery path; they do not imply that the entire website has only one source machine and one cache.

How Does a Request Reach a CDN?

A request reaches a CDN through the site's configured routing, which can involve DNS and network routing decisions. The hostname directs the client toward a CDN endpoint, and the network carries the connection to an appropriate delivery location. The selected location is not necessarily the geographically closest building.

Network topology, service policy, and current routing affect the path. A client in one country and a proxy exit in another can therefore see different delivery behavior. The result should be measured at the application connection rather than inferred from the collector's physical office location.

Once the edge receives the request, it evaluates the host, path, method, headers, and applicable configuration. A static asset and an authenticated account page can follow different handling rules even when they belong to the same website. The CDN's presence alone does not establish whether a particular response is cacheable.

What Happens on a Cache Hit or Miss?

A cache hit occurs when the cache has a suitable stored response that its rules permit it to serve. A miss occurs when the request needs content that cannot be satisfied from an eligible stored response. The edge then usually obtains content from an upstream service, which may be the origin or another cache layer.

1. The edge derives a cache key from the request and configured rules.
2. It looks for a stored response matching that key and relevant variation.
3. It evaluates freshness and any requirements to contact an upstream service.
4. It serves an eligible response or obtains an appropriate upstream response.
5. It stores the new response only when the response and cache policy allow storage.

A miss does not necessarily mean a broken CDN. New content, expired entries, intentionally uncached pages, and uncommon variants can all require upstream work. A high hit ratio is useful only when the served content is correct for the users and requests involved.

Different edge locations may hold different objects. A successful fetch in one location does not prove that every location already has the same stored response. Delivery systems can use several cache tiers, so the precise path depends on the provider and site configuration.

What Determines the Cache Key?

A cache key determines which requests can reuse the same stored representation. It commonly includes the target resource identity and can incorporate additional request characteristics through standard semantics and site-specific configuration.

The HTTP Vary field identifies request fields that affected selection of a representation. If content varies by an accepted encoding or language, those dimensions can matter when selecting a stored response. A cache that ignores an important variation can return the wrong representation.

Query strings require an explicit policy. Some applications use them to identify different resources, while others use them for tracking. Treating every query parameter as irrelevant can merge distinct content. Treating every parameter as a unique key can fragment the cache and reduce reuse.

For a hypothetical regional product page, decide whether currency and market are expressed in the URL, a header, or a cookie. That decision affects both the application and cache configuration. A cache key should reflect the representation's actual identity rather than a guess based on its path alone.

Freshness, Validation, and Storage Are Different

Freshness determines whether a stored response can be reused without checking upstream, while validation checks whether that stored response remains applicable. Storage policy determines whether the response may be kept at all. The HTTP caching rules distinguish these decisions.

A response with no-cache can still be stored, but reuse requires successful validation under the applicable rules. A response with no-store instructs caches not to store it. A private response is restricted from shared-cache storage in the ordinary unqualified case. These directives should not be treated as interchangeable spellings for disabling every cache.

Validators such as entity tags can let an upstream server confirm that a stored representation remains usable. A conditional request may avoid transferring the complete body again when the representation has not changed. Validation still involves an upstream check; it is different from serving a fresh stored response immediately.

The Age field describes an estimate of how long a response has been resident in caches and associated transit. It is not the age of the underlying business fact. A product price may have been unchanged for days even when the response was generated recently, or may have changed at the source after the cached response was created.

How Do Content Updates Reach the Edge?

Content updates reach users through expiration, validation, explicit invalidation, or a new resource identity. Which mechanism applies depends on the website's deployment and cache policies. A published origin change does not automatically imply that every stored edge response has already been removed.

Versioned asset URLs provide a clear identity for a changed file. When a deployment points to a new filename, caches can retain the old object without confusing it with the new one. This approach is useful for assets whose contents should remain stable under one URL.

Frequently changing documents need a different freshness policy. A public announcement page and an account balance page have different requirements, even if both are delivered as HTML. The cache lifetime should follow the acceptable staleness of the content and the privacy properties of the representation.

When investigating an update, compare the URL, response headers, and body version together. A cache-status header can be useful evidence, but its field name and interpretation are provider-specific. Avoid treating one vendor's header vocabulary as a universal HTTP standard.

What Does TLS Termination Change?

A CDN that terminates TLS decrypts the client connection and may create a separate secure connection to the origin. The TLS connection model applies to each secure connection separately.

The edge needs access to the HTTP request to apply content-aware routing and caching. Its connection to the origin has separate trust and transport settings. A valid certificate at the public hostname does not by itself verify the configuration of the origin-facing connection.

For a collector, the public certificate authenticates the expected endpoint under the client's trust rules. It does not establish whether the body came from a cache, which internal service generated it, or whether the page contains the desired records. Those questions require response and content inspection.

Why Regional Checks Can Produce Different Content

Regional checks can differ because the website and delivery system receive different request contexts. Exit location, resolver behavior, language headers, and stored session preferences can all influence the observed representation. A difference should be investigated before it is described as a CDN inconsistency.

The Scrapeless proxy location settings support selecting available exit locations. Keep the selected geography stable within a comparison and record the intended market alongside the observation. A gateway's region is not automatically the exit region used by the destination.

Scrapeless Proxy Solutions can provide the network viewpoint for these checks while your application owns content validation. The related cloud proxy architecture article explains the intermediary role. Consult Scrapeless pricing when estimating the bandwidth of regional asset and page checks.

A useful observation record includes the requested URL, final URL, collection context, response time, and a content identifier or hash when appropriate. A hash proves equality of the bytes being compared, not equality of the user experience; dynamic scripts can still produce different rendered behavior from similar initial documents.

How to Interpret CDN Symptoms

Interpret CDN symptoms by separating connectivity, cache behavior, and content correctness. If an asset loads but a page does not, compare their policies and upstream services. If a response is stale, inspect freshness and validation evidence before changing the collection client.

A response delivered quickly may be an error page stored or generated at the edge. A response delivered slowly may still be correct and newly obtained from the origin. Latency alone cannot classify the result. Use the fields and page markers required by the application.

Likewise, successful CDN delivery of one resource does not prove that an authenticated application view renders correctly. The resource may be public while the view requires credentials, scripts, and other dependencies. Validate the complete user-facing operation when that is the actual requirement.

Conclusion

A CDN routes requests and reuses eligible content under explicit cache rules. Understand the cache key, freshness policy, and TLS boundary before interpreting a response. For regional data work, preserve the collection context and judge success by the returned representation, not delivery speed alone.