A user clicks play on a 4 GB video or downloads an incremental 50 MB update package for your desktop software. Their client sends a standard HTTP request with a Range: bytes=0-1048575 header, requesting just the first megabyte to buffer the start or read the file headers. If your edge caching layer is naively configured, your content delivery network (CDN) forward-proxies that request to your origin object storage by pulling the entire 4 GB archive across the wire, only to throw 99.9% of it away once the user stops streaming.
Multiply that single request across thousands of concurrent clients, media scrubbers, or distributed software updaters, and your egress bandwidth costs explode overnight. Even worse, your origin storage encounters severe I/O exhaustion, driving p99 response times into multi-second territory.
Solving this bottleneck requires a technical understanding of origin range requests cdn caching. When CDNs and reverse proxies handle partial content requests without coordinated sub-range caching, they introduce hidden latency penalties, thundering herd problems on origin object stores, and runaway network egress bills.
In this deep dive, we will analyze why naive range caching breaks, examine how modern proxy architectures coordinate partial fetches, walk through a practical proxy implementation, and map out a hardened runbook for your infrastructure.
The Hidden Mechanics of Naive Byte-Range Requests
To understand why range caching fails, we must inspect how edge proxies handle HTTP Range headers under default configurations.
The standard HTTP specification (RFC 9110) allows clients to request partial representations of a resource using the Range header:
GET /assets/installer-v2.bin HTTP/1.1
Host: downloads.example.com
Range: bytes=0-2097151
When an upstream CDN node or reverse proxy receives this request, it typically follows one of three standard behaviors:
- Pass-Through (No Caching): The proxy simply forwards the
Rangeheader directly to the origin. If 10,000 users scrub through the same video file or load metadata blocks from zip archives, 10,000 distinct range requests hit your origin object store (such as AWS S3, Google Cloud Storage, or Cloudflare R2). - Whole-File Origin Fetching: The proxy converts the partial request into a full
GET /assets/installer-v2.binrequest to the origin, attempting to store the entire 4 GB object on local edge disk. The proxy extracts bytes0-2097151to satisfy the client immediately while continuing to stream the rest in the background. If the user disconnects or pauses after 3 MB, the proxy either aborts the origin stream (wasting the fetch) or finishes downloading gigabytes that nobody requested. - Discrete Slice Fragmentation: The proxy caches the exact byte-range requested (
bytes 0-2097151) as an isolated cache key. If a subsequent client requestsbytes 1048576-3145727(overlapping by 1 MB), the proxy registers a cache miss because the exact requested byte window does not match any cached entity. It hits the origin again, creating duplicated, overlapping fragments on edge disks.
The Real Cost: A Hypothetical Architecture Breakdown
To illustrate how quickly naive caching drains infrastructure budgets, consider a realistic, hypothetical engineering scenario:
- You operate a software platform distributing game asset bundles or AI model weights.
- Your catalog contains 2,000 files, each averaging 1.5 GB.
- Clients make 500,000 range requests per day, primarily reading 2 MB chunks (e.g., streaming audio, reading asset headers, or downloading package manifests).
- Origin storage egress costs an industry-standard $0.09 per gigabyte.
Under Strategy 2 (Whole-File Fetching), if 10% of those requests trigger cold edge nodes that initiate full-file downloads (50,000 full downloads × 1.5 GB), your origin egress transfers 75,000 GB (75 TB) per day. That equates to roughly $6,750 per day in origin egress alone, even though the actual data consumed by end clients was only 1,000 GB (1 TB).
Under Strategy 3 (Discrete Slice Fragmentation), edge disks quickly fill up with non-reusable fragments. Cache hit ratios plummet below 25%, forcing constant eviction loops and saturating the origin's p99 latency. Similar architecture bottlenecks frequently emerge in asynchronous pipelines, much like how unpartitioned queues cause contention when isolating consumer tests on a shared message broker.
How Intelligent Origin Range Request Coordination Works
Modern edge architectures, such as Cloudflare's Rust-based Pingora framework, high-performance NGINX installations, or custom Envoy sidecars, solve this problem through synchronized slice fetching and origin range conversion.
Instead of guessing what the client wants or pulling whole multi-gigabyte blobs, an optimized edge proxy standardizes byte requests into uniform, fixed-size chunks (typically 1 MB, 2 MB, or 8 MB slices).
Client Request:
[ Range: bytes=1.5MB - 3.2MB ]
│
▼
Edge Proxy Layer
(Normalizes into Slices)
│
┌────────┴────────┐
▼ ▼
Slice 1 (1MB-2MB) Slice 2 (2MB-3MB) Slice 3 (3MB-4MB)
[Cache HIT] [Cache MISS] [Cache MISS]
│ │
▼ ▼
Fetch from Origin Fetch from Origin
(Range: 2MB-3MB) (Range: 3MB-4MB)
Here is the operational lifecycle of an intelligent range-coordinating proxy:
- Alignment and Normalization: When a client requests an arbitrary range (for instance,
bytes=1572864-3670015), the proxy breaks the range down into normalized slice boundaries (e.g., Slice A:1MB-2MB, Slice B:2MB-3MB, Slice C:3MB-4MB). - Individual Slice Inspection: The proxy inspects its local cache for each discrete slice.
- Targeted Upstream Range Requests: For any slice not currently cached, the proxy sends an independent partial request (
Range: bytes=start-end) to the origin object storage. The origin never transfers the whole file; it only streams the missing 1 MB or 2 MB segments. - Upstream Request Collapsing (Coalescing): If ten concurrent clients simultaneously request
bytes 0-1048575on a cold cache, the proxy forwards exactly one range request to the origin. The remaining nine requests block asynchronously and read from the incoming stream as bytes arrive. - Client Response Assembly: The proxy stitches the slices together into a single continuous HTTP 206 Partial Content stream, returning the exact range requested by the client.
This design delivers two vital metrics: it shrinks origin egress to match client consumption almost 1:1, and it reduces p99 tail latency because origin storage nodes only have to deliver small, highly parallelizable block streams.
Implementing Slice-Based Range Caching: A Practical Proxy Example
While enterprise CDNs often handle slice caching internally, you can implement identical mechanics within your own application perimeter using reverse proxies or lightweight microservices.
Below is a complete, runnable example using Node.js and TypeScript. It demonstrates how a custom edge gateway or streaming microservice normalizes incoming client range requests into consistent 1 MB chunks, checks an in-memory/disk cache, fetches missing segments from an origin S3/R2 bucket using standard Range headers, and streams the assembled result back to the client.
import http from 'node:http';
import https from 'node:https';
import { URL } from 'node:url';
const CHUNK_SIZE = 1024 * 1024; // 1 MB normalized slice
const ORIGIN_BASE_URL = 'https://origin-storage.internal.example.com';
const memoryCache = new Map<string, Buffer>();
interface RangeSpan {
start: number;
end: number;
}
function parseRangeHeader(rangeHeader: string | undefined, totalSize: number): RangeSpan | null {
if (!rangeHeader || !rangeHeader.startsWith('bytes=')) return null;
const parts = rangeHeader.replace('bytes=', '').split('-');
const start = parseInt(parts[0], 10);
const end = parts[1] ? parseInt(parts[1], 10) : totalSize - 1;
if (isNaN(start) || isNaN(end) || start > end || start >= totalSize) {
return null;
}
return { start, end: Math.min(end, totalSize - 1) };
}
async function fetchSliceFromOrigin(assetPath: string, sliceIndex: number): Promise<Buffer> {
const cacheKey = `${assetPath}_slice_${sliceIndex}`;
if (memoryCache.has(cacheKey)) {
return memoryCache.get(cacheKey)!;
}
const sliceStart = sliceIndex * CHUNK_SIZE;
const sliceEnd = sliceStart + CHUNK_SIZE - 1;
const originUrl = new URL(assetPath, ORIGIN_BASE_URL);
return new Promise((resolve, reject) => {
const req = https.get(
originUrl,
{
headers: {
Range: `bytes=${sliceStart}-${sliceEnd}`,
},
},
(res) => {
if (res.statusCode !== 206 && res.statusCode !== 200) {
reject(new Error(`Origin returned status code: ${res.statusCode}`));
return;
}
const chunks: Buffer[] = [];
res.on('data', (d) => chunks.push(d));
res.on('end', () => {
const completeBuffer = Buffer.concat(chunks);
memoryCache.set(cacheKey, completeBuffer);
resolve(completeBuffer);
});
}
);
req.on('error', reject);
});
}
const server = http.createServer(async (req, res) => {
const url = new URL(req.url || '/', `http://${req.headers.host}`);
const assetPath = url.pathname;
// In production, fetch or query metadata from Redis, database, or HEAD request
const totalFileSize = 50 * 1024 * 1024; // Simulated 50 MB file
const range = parseRangeHeader(req.headers.range, totalFileSize);
if (!range) {
res.writeHead(416, { 'Content-Range': `bytes */${totalFileSize}` });
res.end();
return;
}
const firstSliceIndex = Math.floor(range.start / CHUNK_SIZE);
const lastSliceIndex = Math.floor(range.end / CHUNK_SIZE);
res.writeHead(206, {
'Content-Range': `bytes ${range.start}-${range.end}/${totalFileSize}`,
'Accept-Ranges': 'bytes',
'Content-Length': range.end - range.start + 1,
'Content-Type': 'application/octet-stream',
});
for (let idx = firstSliceIndex; idx <= lastSliceIndex; idx++) {
try {
const sliceBuffer = await fetchSliceFromOrigin(assetPath, idx);
const sliceStartByte = idx * CHUNK_SIZE;
const sliceEndByte = sliceStartByte + sliceBuffer.length - 1;
// Calculate byte offsets within this specific slice
const extractStart = Math.max(0, range.start - sliceStartByte);
const extractEnd = Math.min(sliceBuffer.length, range.end - sliceStartByte + 1);
const subChunk = sliceBuffer.subarray(extractStart, extractEnd);
res.write(subChunk);
} catch (err) {
console.error(`Failed to pipe slice ${idx}:`, err);
res.destroy();
return;
}
}
res.end();
});
server.listen(8080, () => {
console.log('Range caching proxy listening on port 8080');
});
Key Implementation Principles
- Deterministic Cache Keys: Each slice is keyed by its file path and its normalized slice index (
${assetPath}_slice_${sliceIndex}). Regardless of whether a client requests bytes 100 to 5,000 or bytes 4,000 to 8,000, both hit slice index 0. - Memory Bounds: In production environments, replace the simple in-memory
Mapwith an LRU cache or local NVMe storage using consistent hashing across proxy worker threads. - Subarray Windowing: By buffering uniform 1 MB slices and taking
.subarray(extractStart, extractEnd), the proxy avoids extraneous string allocations and memory copies.
Production Pitfalls: Where Range Caching Fails
Engineering an edge architecture that coordinates origin range requests introduces subtle concurrency and cache invalidation challenges. Teams frequently encounter four significant traps:
1. Inconsistent ETag and Last-Modified Desynchronization
When assembling slices dynamically, the origin must guarantee that the file did not mutate between the fetch of Slice 0 and Slice 10. If an updated file is deployed while a client streams sequential slices, the client will receive bytes from version A stitched directly to bytes from version B, resulting in binary corruption.
- Fix: Enforce strict validation using the
If-Rangeheader or compare the originETagon every fetched slice. If an ETag mismatch occurs mid-flight, immediately flush all related local slices and restart the client download stream.
2. Cache Thundering Herds on Cold Deployments
When releasing a major update (such as a 10 GB installer or game patch), hundreds of edge nodes will simultaneously miss their local caches. If request collapsing is missing, every edge node issues thousands of parallel range requests to your origin storage.
- Fix: Enable request coalescing (also known as
proxy_cache_use_stale updatingin NGINX or lock coordination in modern proxy engines). The proxy should execute only one origin fetch per missing slice while queuing remaining client requests on that exact slice promise.
3. Sub-Optimal Slice Size Allocation
Choosing the wrong slice size harms performance on both extremes:
- Too small (e.g., 64 KB): Introduces excessive HTTP request overhead, connection round-trips, and header processing between the edge and the origin.
- Too large (e.g., 32 MB): Causes streaming latency spikes for video scrubbers and wastes egress bandwidth when clients only read metadata headers.
- Rule of thumb: Benchmark client read patterns. For media streaming and random-access binary assets, slice sizes between 1 MB and 4 MB generally yield the best compromise between latency and bandwidth efficiency.
Engineering robust, resilient backend infrastructure often exposes unexpected scaling bottlenecks across cloud networks and game engines. Teams building heavy-duty platforms often consult with an experienced custom software development partner to design fault-tolerant streaming pipelines, optimize edge delivery, and prevent surprise egress invoices.
Step-by-Step Runbook: Hardening Origin Range Caching
Follow this five-step runbook to diagnose, configure, and verify byte-range caching across your infrastructure.
Step 1: Audit Current Behavior via cURL
│
Step 2: Configure Edge Slicing Rules
│
Step 3: Harmonize Origin Headers (ETags & Cache-Control)
│
Step 4: Verify Origin Egress Metrics & Cache Hits
│
Step 5: Run Automated Invalidation and Concurrency Tests
Step 1: Audit Current Edge Behavior
Verify whether your current proxy or CDN is caching full assets, passing ranges through, or returning partial chunks properly. Execute two consecutive range requests for the same byte range:
curl -svo /dev/null \
-H "Range: bytes=0-1048575" \
https://downloads.example.com/assets/bundle.pkg
Inspect the response headers:
- Ensure
Accept-Ranges: bytesis present. - Verify
HTTP/1.1 206 Partial Content(or HTTP/2 equivalent) is returned. - Inspect the cache status header (e.g.,
cf-cache-status,x-cache, orage). On the second request, the cache status should showHIT.
Next, run a request for an overlapping range (e.g., bytes=524288-1572863). If your cache status shows a MISS and origin metrics reflect a full asset pull, your proxy does not support normalized slice slicing.
Step 2: Configure Edge Slicing Rules
If using standard NGINX as your reverse proxy, enable the native slice module to coordinate partial requests automatically:
http {
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=range_cache:100m
max_size=50g inactive=7d use_temp_path=off;
server {
listen 80;
server_name downloads.example.com;
location /assets/ {
slice 1m;
proxy_cache range_cache;
proxy_cache_key $uri$is_args$args$slice_range;
# Forward the slice range to the origin
proxy_set_header Range $slice_range;
# Collapse multiple requests for the same slice
proxy_cache_lock on;
proxy_cache_lock_timeout 5s;
# Origin settings
proxy_pass https://storage.internal.example.com;
proxy_http_version 1.1;
proxy_set_header Connection "";
# Cache partial content
proxy_cache_valid 206 200 30d;
}
}
}
Notice the critical variable: $slice_range. NGINX automatically intercepts the client's arbitrary Range header, splits it into 1 MB chunks, populates $slice_range, and stores each slice under its own deterministic cache key.
Step 3: Harmonize Origin Headers
Your origin storage bucket or backend application must provide reliable cache validators:
- Strong ETags: Ensure ETags are consistent across all origin replicas. Avoid dynamic gzip/brotli compression on binary assets, as dynamic compression can alter the ETag between requests.
- Cache-Control: Ensure the origin returns
Cache-Control: public, max-age=2592000, immutablefor static release bundles.
Step 4: Verify Origin Egress Metrics
Monitor your origin bucket's egress bandwidth and operations metrics during high-traffic windows:
- Byte Out vs. Client Bytes: Sum total client-downloaded gigabytes versus origin egress gigabytes. In a healthy slice-caching architecture, origin egress should represent only a fraction of total edge egress (typically below 5-10% on popular assets).
- HTTP 206 Ratio: Ensure the origin logs show almost exclusively
206 Partial Contentstatus codes rather than200 OKfull-file responses.
Step 5: Run Invalidation and Concurrency Tests
Simulate a concurrent release using an HTTP benchmarking tool like k6 or wrk. Send 100 concurrent requests across randomized byte offsets. Observe origin connections: you should see origin requests scale linearly with the unique slices requested, never with the total number of connected clients.
Best Practices for Large Asset Delivery
To keep latency low and cloud hosting budgets predictable, incorporate these four best practices into your infrastructure design:
- Align Slice Boundaries with Upstream Storage Block Sizes: Cloud object stores read data in distinct physical blocks. Aligning your proxy slice size (e.g., 2 MB or 4 MB) with common object store block alignments minimizes internal I/O amplification on the storage cluster.
- Isolate Range-Cached Assets on Dedicated Hostnames: Keep slice-cached assets on separate subdomains (e.g.,
cdn-assets.example.comvsapi.example.com). This isolates cache-lock timeouts, large disk allocations, and proxy memory pools from latency-sensitive JSON microservice traffic. - Implement Pre-Warming for Critical Slices Only: You do not need to pre-warm an entire 10 GB file across your global edge network. When publishing a build, send automated
Range: bytes=0-2097151requests to your primary edge locations. Pre-warming just the first 2 MB ensures instant time-to-first-byte (TTFB) for users while deferring subsequent chunks until demanded. - Enforce Request Coalescing at Every Layer: Never disable cache locks or request coalescing on partial content. Without request collapsing, the cache stampede from a single viral asset release can overwhelm origin storage rate limits (such as S3 503 SlowDown errors). If your team is architecting live multiplayer services or complex content delivery pipelines, reviewing practical architectural lessons from building dedicated game backends provides valuable perspective on managing edge workloads and high-throughput assets.
Wrapping Up
Delivering multi-gigabyte files does not have to result in runaway egress costs or degraded edge latency. Naive byte-range handling wastes origin bandwidth by downloading unnecessary data or failing to reuse cached fragments.
By deploying normalized slice caching, implementing request coalescing, and standardizing origin range headers, you turn unpredictable partial-content traffic into a deterministic, high-efficiency streaming pipeline.
Auditing your proxy configuration and testing your edge response headers today is the easiest way to catch egress leaks before your next major release. Weighing whether to optimize your infrastructure in-house or need an expert audit for your next release? See how we approach high-performance custom software development to keep your delivery fast, scalable, and cost-effective.
Source: Origin Range Requests: Serving large files without range-request tuning