Uploading with ReadableStream Request Bodies #

Pass a ReadableStream as the body of a fetch and add duplex: "half", and the browser will send bytes as you produce them instead of buffering the whole payload first — but only over HTTP/2 or HTTP/3, only on Chromium, and only to a server that tolerates a request with no Content-Length.

This article sits inside the Streams API for uploads topic within upload fundamentals and browser APIs. It covers the request itself: the flags, the transport constraints, and the fallback you must ship alongside it.

When to use this approach #

• You are generating bytes rather than holding them — encrypting, transcoding, hashing, or concatenating sources — and materialising the whole result as a Blob first would blow your memory budget.
• You want the first byte on the wire before the last byte exists, so a 4 GB export starts uploading while it is still being written.
• You control the receiving endpoint. If you are uploading to a signed object-storage URL, stop here: S3 rejects a PUT with no Content-Length, and chunked slicing via Blob.slice is the right tool instead.

If you simply want to post a file that already exists on disk, body: file already streams from disk without buffering — use the ordinary fetch and FormData path and skip all of the constraints below.

Prerequisites #

1. Chromium 105 or newer. Firefox and Safari still ignore duplex and stringify the stream, so a fallback is not optional.
2. An HTTPS origin served over HTTP/2 or HTTP/3 end to end — including every reverse proxy in front of your app.
3. A server that reads a request body of unknown length (Node’s http/http2, Go, most modern frameworks) rather than one that demands Content-Length up front.
4. TypeScript with lib: ["DOM", "ES2022"].

Implementation #

The block below is the whole pattern: a pull-based stream over a Blob, a feature probe that is honest about partial support, and an uploader that silently degrades to sending the Blob itself.

/** Pull-based stream over a Blob: a slice is read only when the sink asks for it. */
export function blobStream(blob: Blob, chunkSize = 256 * 1024): ReadableStream<Uint8Array> {
  let offset = 0;
  return new ReadableStream<Uint8Array>(
    {
      async pull(controller) {
        if (offset >= blob.size) {
          controller.close();
          return;
        }
        const end = Math.min(offset + chunkSize, blob.size);
        const buf = await blob.slice(offset, end).arrayBuffer(); // reads 256 KB, not 4 GB
        offset = end;
        controller.enqueue(new Uint8Array(buf));
      },
      cancel(reason) {
        console.warn("upload stream cancelled:", reason);
      },
    },
    // Keep at most 2 chunks (512 KB) queued ahead of the network.
    new CountQueuingStrategy({ highWaterMark: 2 }),
  );
}

type StreamInit = RequestInit & { duplex?: "half" };

/** True only on engines that really accept a ReadableStream request body. */
export const supportsRequestStreams: boolean = (() => {
  let duplexAccessed = false;
  try {
    const probeInit: StreamInit = {
      method: "POST",
      body: new ReadableStream(),
      get duplex() {
        duplexAccessed = true;
        return "half" as const;
      },
    };
    const probe = new Request("https://probe.invalid/", probeInit);
    // Engines without support stringify the stream and stamp text/plain on it.
    return duplexAccessed && !probe.headers.has("Content-Type");
  } catch {
    return false; // no Request/ReadableStream (SSR), or the constructor threw
  }
})();

export interface StreamUploadOptions {
  url: string;
  file: File;
  chunkSize?: number;
  signal?: AbortSignal;
}

export async function uploadStreaming(opts: StreamUploadOptions): Promise<Response> {
  const { url, file, chunkSize = 256 * 1024, signal } = opts;

  const init: StreamInit = {
    method: "PUT",
    headers: { "Content-Type": file.type || "application/octet-stream" },
    signal,
    // A stream body cannot be replayed, so a 307/308 must fail loudly, not silently.
    redirect: "error",
  };

  if (supportsRequestStreams) {
    init.body = blobStream(file, chunkSize);
    init.duplex = "half"; // mandatory whenever body is a ReadableStream
  } else {
    init.body = file; // Blob path: the browser sets Content-Length for us
  }

  const res = await fetch(url, init);
  if (!res.ok) {
    throw new Error(`stream upload failed: HTTP ${res.status} ${res.statusText}`);
  }
  return res;
}

Line-by-line on the critical parts #

• pull(controller) instead of start(controller). pull is invoked only when the internal queue drops below the high-water mark. With CountQueuingStrategy({ highWaterMark: 2 }) and 256 KB chunks, at most 512 KB of file data is resident at any moment, regardless of whether the file is 4 MB or 4 GB. Put the same logic in start and you loop the entire file into memory immediately — the exact bug streaming was meant to avoid.
• blob.slice(offset, end) is free. It records offsets; no bytes move until arrayBuffer() resolves. That is why the read happens inside pull and not in a precomputed array.
• duplex: "half" declares that you will finish sending the request before you read the response. It is the only accepted value today; "full" throws. Omitting it is a TypeError at fetch() time, not a silent downgrade.
• redirect: "error" converts an unfollowable redirect into an immediate, named failure. The default "follow" also fails on a non-303 redirect (the body is gone), but with a generic network error that is far harder to diagnose.
• init.body = file in the fallback branch is not a compromise for correctness, only for latency: the browser still streams from disk and never buffers, it just knows the length in advance.
• The probe checks two things. duplexAccessed alone is not enough, because a getter is read by any engine that enumerates the init object. Non-supporting engines coerce the stream to the string "[object ReadableStream]" and set Content-Type: text/plain;charset=UTF-8; the absence of that header is the real signal.

file.stream() gives you the same shape in one call, and it is the right choice when you have nothing to do with the bytes on the way past. Chromium reads it in 64 KiB chunks that you cannot configure. Write the stream yourself when you need a larger chunk size to cut per-chunk overhead, when you want to count bytes for a progress readout, or when you are splicing several sources into one body.

Why duplex: "half" is mandatory #

Before streaming bodies existed, every body value had a known length and a fixed source, so a Request was replayable. A stream is neither. duplex is the explicit acknowledgement that you understand the request is now a one-shot pipe, and the specification requires it on any request whose body is a ReadableStream. There is no default.

Omit it in Chromium and fetch() throws synchronously before a packet leaves:

TypeError: Failed to execute 'fetch' on 'Window':
  The `duplex` member must be specified for a request with a streaming body

Node’s undici — which matters when you share upload code between the browser and a server-side proxy — throws a differently worded version of the same rule:

TypeError: RequestInit: duplex option is required when sending a body.

"half" means the request finishes before the response begins. Full duplex, where you read response bytes while still sending, is not implemented in any browser; duplex: "full" throws a RangeError on the value. Half duplex is enough for uploads: you send the file, then read the JSON receipt.

The transport requirement #

Chromium refuses to send a streamed body over HTTP/1.1. The connection is negotiated first, and if ALPN lands on http/1.1 the request is aborted rather than downgraded to chunked transfer encoding. The fetch() promise rejects with a bare TypeError: Failed to fetch, and the DevTools Network panel shows the real cause in the status column:

(failed) net::ERR_H2_OR_QUIC_REQUIRED

This bites hardest in development. A plain node server.js on http://localhost:3000 is HTTP/1.1, so the code that works in production fails on your machine. There is no localhost exemption. The practical fix is to put a TLS-terminating proxy that speaks h2 in front of your dev server.

No Content-Length, and who refuses it #

A stream has no length, and Content-Length is a forbidden header name in fetch — you cannot supply it yourself even if you know the size. Over HTTP/2 the body simply arrives as a sequence of DATA frames terminated by END_STREAM, with no length declared anywhere. On the HTTP/1.1 hop between a proxy and your origin, that becomes Transfer-Encoding: chunked.

Three categories of receiver react badly:

The last row is the target you want, and the receiving side of it — piping an unbounded request body straight to disk or object storage without buffering — is the subject of streaming file uploads in Node.js with Web Streams.

The S3 case is the one that catches people who read about streaming and try to point it at a presigned URL from AWS SDK v3. Object storage needs the length to allocate the object, so a streamed body is rejected outright. For direct-to-storage uploads of anything large, the chunked approach in handling 500 MB file uploads is what you want; streaming request bodies are for your own endpoints.

Nginx’s default is the subtler trap, because nothing errors — throughput just collapses. With proxy_request_buffering on (the default), nginx accumulates the entire request into client_body_temp_path before opening the upstream connection, so your carefully streamed 3 GB body becomes a 3 GB disk write followed by a normal upload. Turn buffering off on the upload location only.

Memory: what streaming actually buys you #

The win is not bandwidth, it is resident memory and time-to-first-byte. Reading a file into an ArrayBuffer before sending it is the pattern most upload code starts with, and it scales linearly with file size until the tab is killed. A pull-based stream is flat.

Time-to-first-byte matters just as much when the body is generated. If you encrypt a 1.2 GB file before upload, the buffered version spends 40-plus seconds on CPU with an idle socket, then uploads. The streamed version overlaps the two and finishes in roughly the longer of the two durations rather than their sum.

Retries, aborts, and progress #

A ReadableStream is single-use. Once fetch has started reading it the stream is disturbed, and passing it to a second fetch fails at construction:

TypeError: Failed to construct 'Request':
  Cannot construct a Request with a ReadableStream body that is disturbed or locked

This is why blobStream(file) is a factory rather than a value. Every retry attempt must call it again to get a fresh stream with offset reset to zero; the underlying File is re-readable from disk, so nothing is lost. Wire that into whatever backoff policy you already use from browser timeout and retry logic, and be aware that a streamed request always restarts from byte zero — there is no range resume, which is the trade-off against the chunked designs in resuming uploads after network loss.

Aborting works as usual: pass a signal, and controller.abort() both rejects the fetch with an AbortError and invokes your stream’s cancel(reason) callback so you can release file handles.

Progress is the pleasant surprise. fetch still has no upload progress event, but you own the producer, so you can count bytes as you enqueue them — or splice a counting stage into the pipeline, which is what tracking upload progress with a TransformStream does properly. Either way the count is accurate for “handed to the network stack”, which leads the acknowledged byte count by roughly one congestion window — good enough for a bar, misleading for a completion claim. Feeding those numbers to a live UI is covered by real-time upload progress events.

Configuration gotchas #

Missing duplex. Chromium throws TypeError: Failed to execute 'fetch' on 'Window': The 'duplex' member must be specified for a request with a streaming body before any network activity. Add duplex: "half". If TypeScript rejects the property, your lib.dom is older than the spec change — widen the type with RequestInit & { duplex?: "half" } as shown above.

HTTP/1.1 endpoint. The fetch rejects with TypeError: Failed to fetch and DevTools reports net::ERR_H2_OR_QUIC_REQUIRED. Serve the endpoint over TLS with h2 negotiated by ALPN. Check the Network panel’s Protocol column reads h2, not http/1.1.

S3 or GCS as the target. The response is HTTP/1.1 411 Length Required with <Code>MissingContentLength</Code><Message>You must provide the Content-Length HTTP header.</Message>. There is no header you can add — fetch forbids setting Content-Length. Send the file as a Blob or as multipart parts instead.

Nginx swallowing the stream. Symptoms are correct uploads with zero streaming benefit, plus 413 Request Entity Too Large and client intended to send too large body in the error log. Set client_max_body_size 0; and proxy_request_buffering off; inside the upload location, and keep proxy_http_version 1.1;.

Reusing a stream on retry. Cannot construct a Request with a ReadableStream body that is disturbed or locked. Build a new stream per attempt.

Verification #

Prove three things: the header is absent, the bytes arrive intact, and the protocol is h2. Start a receiver that reports what it saw:

import { createServer } from "node:http";
import { createHash } from "node:crypto";

createServer((req, res) => {
  const hash = createHash("sha256");
  let bytes = 0;
  console.log("content-length:", req.headers["content-length"] ?? "(absent)");
  console.log("transfer-encoding:", req.headers["transfer-encoding"] ?? "(none)");
  req.on("data", (chunk: Buffer) => {
    bytes += chunk.length;
    hash.update(chunk);
  });
  req.on("end", () => {
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ bytes, sha256: hash.digest("hex") }));
  });
}).listen(3000);

Put an h2 front end on it, because the browser will not stream to port 3000 directly:

# Terminates TLS with a locally-trusted cert and speaks HTTP/2 to the browser.
caddy reverse-proxy --from https://localhost:8443 --to http://localhost:3000

Then run the client assertion from a page served over the same origin:

console.assert(supportsRequestStreams, "this engine cannot stream request bodies");

const file = new File([new Uint8Array(4 * 1024 * 1024)], "four-mib.bin");
const res = await uploadStreaming({ url: "https://localhost:8443/upload", file });
const body = (await res.json()) as { bytes: number; sha256: string };

console.assert(body.bytes === file.size, `expected ${file.size} bytes, got ${body.bytes}`);
console.log("streamed upload verified:", body.sha256);

The server log should print content-length: (absent). If it prints a number, the streaming branch did not run — either the probe returned false or a proxy buffered and re-framed the request.

Frequently Asked Questions