Computing File Checksums in the Browser with Web Crypto #

crypto.subtle.digest("SHA-256", buffer) only hashes bytes you already hold in memory, so checksumming a large file means slicing it with Blob.slice and then — because SubtleCrypto has no update()/final() pair — either folding the per-chunk digests into a root hash or running a JavaScript SHA-256 incrementally inside a Worker.

This article sits in the File API and Blob objects topic within upload fundamentals and browser APIs. It assumes you already know how to get a File reference and how to read its bytes.

When to use this approach #

• You want content-addressed deduplication: hash first, ask the server whether it already has that digest, and skip the transfer entirely when it does.
• You need end-to-end integrity on a direct-to-storage upload, so S3 can reject a corrupted body instead of silently storing it.
• You are resuming a partially uploaded file and must prove the local bytes still match what the server holds.

If none of those apply, do not hash. Hashing a 2 GB file costs roughly four to five seconds of CPU on a laptop and far more on a mid-range Android phone; that is a real cost you should only pay for a real benefit.

Prerequisites #

1. A secure context — https://, localhost, or 127.0.0.1. crypto.subtle is undefined everywhere else.
2. A File or Blob reference and, for large files, the byte-range slicing described in slicing large files with Blob.slice.
3. TypeScript with lib: ["DOM", "DOM.Iterable", "ES2022"], and a bundler that understands new Worker(new URL(...), { type: "module" }) — Vite, esbuild, webpack 5 and Parcel all do.

Why SubtleCrypto has no streaming digest #

Web Crypto exposes exactly one digest entry point: crypto.subtle.digest(algorithm, data), where data is a BufferSource. There is no hasher object, no update(), no final(). The practical ceiling on a one-shot digest is therefore the largest ArrayBuffer you can allocate: Chrome will hash a 200 MB buffer without complaint, but await file.arrayBuffer() on a 3 GB video rejects, and iOS Safari kills the tab well before that.

Other runtimes filled the gap in mutually incompatible ways. Node has crypto.createHash("sha256") with update()/digest(). Cloudflare Workers ships a non-standard crypto.DigestStream you can pipe a ReadableStream into. Neither exists in a browser tab. That leaves three shapes, and they do not all produce the same number:

Web Crypto supports SHA-1, SHA-256, SHA-384 and SHA-512 and nothing else. There is no MD5, no CRC32, no SHA-3, no BLAKE3. That single fact decides which S3 integrity header you can realistically use, as covered below.

Implementation #

This module covers the two Web Crypto paths: a one-shot digest for small files, and a chunked per-part digest with a composite root for large ones. Both return the digest in the three encodings you actually need.

// checksum.ts
const CHUNK_SIZE = 8 * 1024 * 1024; // 8 MB — see the throughput chart below

export interface FileChecksum {
  hex: string; // 64 lowercase chars — dedupe keys, log lines, DB columns
  base64: string; // 44 chars — the wire format for x-amz-checksum-sha256
  bytes: Uint8Array; // the raw 32 bytes, for building a composite root
}

function toHex(buffer: ArrayBuffer): string {
  return Array.from(new Uint8Array(buffer), (b) => b.toString(16).padStart(2, "0")).join("");
}

function toBase64(buffer: ArrayBuffer): string {
  const bytes = new Uint8Array(buffer);
  let binary = "";
  // Chunk the spread: String.fromCharCode(...) blows the stack past ~65k args.
  for (let i = 0; i < bytes.length; i += 0x8000) {
    binary += String.fromCharCode(...bytes.subarray(i, i + 0x8000));
  }
  return btoa(binary);
}

function encode(digest: ArrayBuffer): FileChecksum {
  return { hex: toHex(digest), base64: toBase64(digest), bytes: new Uint8Array(digest) };
}

function assertSubtle(): SubtleCrypto {
  const subtle = globalThis.crypto?.subtle;
  if (!subtle) {
    throw new Error(
      `crypto.subtle is unavailable (isSecureContext=${globalThis.isSecureContext}) — ` +
        "serve over https:// or localhost",
    );
  }
  return subtle;
}

/** One-shot digest. Peak memory equals file.size — keep this under ~50 MB. */
export async function digestWholeFile(file: Blob): Promise<FileChecksum> {
  const subtle = assertSubtle();
  return encode(await subtle.digest("SHA-256", await file.arrayBuffer()));
}

/** Per-chunk digests plus a composite root. Peak memory equals chunkSize. */
export async function digestByParts(
  file: Blob,
  chunkSize = CHUNK_SIZE,
  onProgress?: (bytesDone: number, total: number) => void,
): Promise<{ parts: FileChecksum[]; root: string }> {
  const subtle = assertSubtle();
  const ranges: Array<[number, number]> = [];
  for (let start = 0; start < file.size; start += chunkSize) {
    ranges.push([start, Math.min(start + chunkSize, file.size)]);
  }
  if (ranges.length === 0) ranges.push([0, 0]); // a zero-byte file still has one part

  const parts: FileChecksum[] = [];
  for (const [start, end] of ranges) {
    const buffer = await file.slice(start, end).arrayBuffer();
    parts.push(encode(await subtle.digest("SHA-256", buffer)));
    onProgress?.(end, file.size);
  }

  // S3's composite rule: SHA-256 over the concatenated raw part digests, then "-N".
  const concatenated = new Uint8Array(parts.length * 32);
  parts.forEach((part, i) => concatenated.set(part.bytes, i * 32));
  const rootDigest = await subtle.digest("SHA-256", concatenated);
  return { parts, root: `${toBase64(rootDigest)}-${parts.length}` };
}

Line-by-line on the critical parts #

• subtle.digest("SHA-256", …) takes the algorithm name with the hyphen. Names are matched case-insensitively, so "sha-256" works, but "SHA256" throws NotSupportedError.
• await file.slice(start, end).arrayBuffer() is the whole memory story. slice() allocates nothing; only arrayBuffer() materialises bytes, and only chunkSize of them at a time. The previous chunk’s buffer is garbage after each loop iteration.
• The loop is deliberately sequential. SHA-256 is CPU-bound and single-threaded; issuing eight digest() calls concurrently multiplies peak memory by eight and buys nothing.
• toBase64 slices at 0x8000 because String.fromCharCode(...bytes) on a 32 KB-plus array throws RangeError: Maximum call stack size exceeded. For a 32-byte digest the loop runs once, but the same helper gets reused on larger payloads.
• parts.length * 32 hard-codes the SHA-256 digest length. Switch to SHA-512 and this becomes 64 — a silent corruption if you forget.
• The -N suffix on root is not decoration: it is exactly the format S3 returns for a multipart object’s x-amz-checksum-sha256, where N is the part count.

Wire it to an input like this:

const input = document.querySelector<HTMLInputElement>("#file")!;
input.addEventListener("change", async () => {
  const file = input.files?.[0];
  if (!file) return;
  const started = performance.now();
  const { root } = await digestByParts(file, 8 * 1024 * 1024, (done, total) => {
    console.log(`hashed ${((done / total) * 100).toFixed(0)}%`);
  });
  console.log(`composite ${root} in ${Math.round(performance.now() - started)} ms`);
});

Incremental SHA-256 in a Worker #

When the server computes a plain sha256sum over the whole object, a composite root is useless — you need the real digest without buffering the file. That means a JavaScript SHA-256 with an update() method, run in a Worker so the synchronous compression rounds never touch the UI thread.

// sha256.ts — incremental SHA-256, because SubtleCrypto has no update()/final().
const K = new Uint32Array([
  0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
  0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
  0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
  0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
  0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
  0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
  0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
  0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2,
]);

export class Sha256 {
  private h = new Uint32Array([
    0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
    0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19,
  ]);
  private buf = new Uint8Array(64);
  private w = new Uint32Array(64);
  private bufLen = 0;
  private total = 0;

  update(data: Uint8Array): this {
    this.total += data.length;
    let offset = 0;
    if (this.bufLen > 0) {
      const need = Math.min(64 - this.bufLen, data.length);
      this.buf.set(data.subarray(0, need), this.bufLen);
      this.bufLen += need;
      offset = need;
      if (this.bufLen === 64) {
        this.block(this.buf, 0);
        this.bufLen = 0;
      }
    }
    for (; offset + 64 <= data.length; offset += 64) this.block(data, offset);
    if (offset < data.length) {
      this.buf.set(data.subarray(offset), 0);
      this.bufLen = data.length - offset;
    }
    return this;
  }

  /** Finalises the hash. Call once — it mutates the internal state. */
  digest(): Uint8Array {
    const bits = this.total * 8;
    this.buf[this.bufLen++] = 0x80;
    if (this.bufLen > 56) {
      this.buf.fill(0, this.bufLen);
      this.block(this.buf, 0);
      this.bufLen = 0;
    }
    this.buf.fill(0, this.bufLen, 56);
    const view = new DataView(this.buf.buffer);
    view.setUint32(56, Math.floor(bits / 0x100000000));
    view.setUint32(60, bits >>> 0);
    this.block(this.buf, 0);
    const out = new Uint8Array(32);
    const outView = new DataView(out.buffer);
    for (let i = 0; i < 8; i++) outView.setUint32(i * 4, this.h[i]);
    return out;
  }

  private block(data: Uint8Array, offset: number): void {
    const w = this.w;
    for (let i = 0; i < 16; i++) {
      const p = offset + i * 4;
      w[i] = (data[p] << 24) | (data[p + 1] << 16) | (data[p + 2] << 8) | data[p + 3];
    }
    for (let i = 16; i < 64; i++) {
      const x = w[i - 15];
      const y = w[i - 2];
      const s0 = ((x >>> 7) | (x << 25)) ^ ((x >>> 18) | (x << 14)) ^ (x >>> 3);
      const s1 = ((y >>> 17) | (y << 15)) ^ ((y >>> 19) | (y << 13)) ^ (y >>> 10);
      w[i] = (w[i - 16] + s0 + w[i - 7] + s1) | 0;
    }
    let [a, b, c, d, e, f, g, hh] = this.h;
    for (let i = 0; i < 64; i++) {
      const S1 = ((e >>> 6) | (e << 26)) ^ ((e >>> 11) | (e << 21)) ^ ((e >>> 25) | (e << 7));
      const t1 = (hh + S1 + ((e & f) ^ (~e & g)) + K[i] + w[i]) | 0;
      const S0 = ((a >>> 2) | (a << 30)) ^ ((a >>> 13) | (a << 19)) ^ ((a >>> 22) | (a << 10));
      const t2 = (S0 + ((a & b) ^ (a & c) ^ (b & c))) | 0;
      hh = g; g = f; f = e; e = (d + t1) | 0;
      d = c; c = b; b = a; a = (t1 + t2) | 0;
    }
    const H = this.h;
    H[0] += a; H[1] += b; H[2] += c; H[3] += d;
    H[4] += e; H[5] += f; H[6] += g; H[7] += hh;
  }
}

The Worker reads slices and feeds them in. Note what is not posted across the boundary:

// hash.worker.ts
import { Sha256 } from "./sha256";

const CHUNK = 8 * 1024 * 1024;

self.onmessage = async (event: MessageEvent<{ file: File }>) => {
  const { file } = event.data;
  const hasher = new Sha256();
  for (let start = 0; start < file.size; start += CHUNK) {
    const end = Math.min(start + CHUNK, file.size);
    hasher.update(new Uint8Array(await file.slice(start, end).arrayBuffer()));
    self.postMessage({ type: "progress", done: end, total: file.size });
  }
  const hex = Array.from(hasher.digest(), (b) => b.toString(16).padStart(2, "0")).join("");
  self.postMessage({ type: "done", hex });
};

// main thread
export function hashInWorker(file: File, onProgress: (fraction: number) => void) {
  return new Promise<string>((resolve, reject) => {
    const worker = new Worker(new URL("./hash.worker.ts", import.meta.url), { type: "module" });
    worker.onmessage = (event) => {
      const msg = event.data as { type: string; done?: number; total?: number; hex?: string };
      if (msg.type === "progress") onProgress(msg.done! / msg.total!);
      if (msg.type === "done") {
        worker.terminate();
        resolve(msg.hex!);
      }
    };
    worker.onerror = (err) => {
      worker.terminate();
      reject(new Error(err.message));
    };
    worker.postMessage({ file });
  });
}

Post the File itself, not its bytes. File and Blob are structured-cloneable and the clone is a reference to the same backing storage, so worker.postMessage({ file }) on a 4 GB file costs nothing. Reading the file into an ArrayBuffer on the main thread and transferring that would defeat the entire exercise — and if you transfer a buffer you still hold a view over, the next access throws TypeError: Cannot perform Construct on a detached ArrayBuffer.

One honest caveat: Chromium runs crypto.subtle.digest on a background thread pool, so the native path rarely freezes the UI on its own. The JavaScript implementation above absolutely does — it is roughly three to five times slower than the native digest and fully synchronous. The Worker is not optional decoration.

Choosing a chunk size #

Chunk size trades per-call overhead against peak memory. Below 256 KB you spend more time in promise plumbing and buffer allocation than in the compression function; above about 4 MB the curve is flat.

Default to 8 MB. It sits on the plateau, keeps peak memory bounded on a phone, and — when you are also driving a multipart upload — lets you reuse the same slice boundaries as your parts, so each part’s digest is a by-product of hashing rather than a second read. For the sizing conversation around the upload itself, see best practices for handling 500MB file uploads.

Hashing a 2 GB file at 470 MB/s takes about 4.3 seconds. Report progress, and cache the result: key an IndexedDB record on ${file.name}:${file.size}:${file.lastModified} so a page reload does not re-hash. The same store already holds your upload state in IndexedDB, which makes resuming after network loss instant instead of a four-second stall.

Using the hash: dedupe and S3 integrity headers #

The digest is 32 raw bytes. What you send depends entirely on the encoding, and mixing them up is the single most common failure here.

Content-MD5 is the header most tutorials reach for, and Web Crypto cannot produce it: there is no MD5 in SubtleCrypto, so you would have to ship a JavaScript MD5 alongside everything else. Use x-amz-checksum-sha256 instead. S3 has supported it since 2022, it is stronger, and you already have the value.

The catch is that the header must be signed. A presigned PUT signed without it returns 403 SignatureDoesNotMatch the moment you add it, so the digest has to reach your signing endpoint before the URL is issued — which is exactly the round trip that also answers “do you already have this file?”.

export async function uploadDeduplicated(file: File): Promise<string> {
  const { hex, base64 } = await digestWholeFile(file);

  // One call does dedupe and signing: the server signs x-amz-checksum-sha256
  // into the URL, so S3 verifies the body it receives.
  const res = await fetch("/api/uploads/intent", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ sha256: hex, size: file.size, contentType: file.type }),
  });
  if (!res.ok) throw new Error(`intent failed: HTTP ${res.status}`);
  const intent = (await res.json()) as { key: string; url?: string };

  if (!intent.url) return intent.key; // server already has these bytes — nothing to send

  const put = await fetch(intent.url, {
    method: "PUT",
    headers: {
      "Content-Type": file.type || "application/octet-stream",
      "x-amz-checksum-sha256": base64,
    },
    body: file,
  });
  if (!put.ok) throw new Error(`upload failed: HTTP ${put.status}`);
  return intent.key;
}

Server side, pass ChecksumSHA256: base64 on the PutObjectCommand before presigning — see generating secure presigned URLs with AWS SDK v3. Treat the client hash as a hint, never as proof: a browser can send any 32 bytes it likes, so keep doing real file signature validation with libmagic after the object lands. What the header buys you is protection against corruption in transit, not against a hostile client.

Configuration gotchas #

TypeError: Cannot read properties of undefined (reading 'digest'). You are on an insecure origin — typically http://192.168.1.20:5173 while testing on a phone. Firefox words it as TypeError: crypto.subtle is undefined. crypto.getRandomValues() still works there, which makes the failure look random. Check globalThis.isSecureContext and serve over HTTPS or a localhost tunnel.

NotSupportedError: Unrecognized name. Thrown by digest() when the algorithm string is not one of SHA-1, SHA-256, SHA-384, SHA-512. The usual culprits are "SHA256" without the hyphen and "MD5", which Web Crypto does not implement at all.

400 Bad Request with <Code>BadDigest</Code> — “The SHA256 you specified did not match what we received.” Either you sent hex where base64 belongs (S3 answers InvalidRequest: Value for x-amz-checksum-sha256 header is invalid for that), or you hashed the original File and uploaded a transformed copy — a resized image or a re-encoded video is different bytes. Hash whatever you actually put in the request body. If you need to read the checksum back off the response, expose it via CORS, or you will hit the same wall described in fixing CORS preflight errors on S3 uploads.

NotReadableError: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired. The user moved, renamed or re-saved the file between hashing and uploading. Chunked hashing widens that window from milliseconds to seconds. Re-check file.size and file.lastModified immediately before the PUT, and surface a “the file changed, please pick it again” message rather than uploading bytes that no longer match the digest you just promised.

Verification #

Compare against the command line. Both encodings come from the same digest, so both must match:

printf 'test' > sample.txt

shasum -a 256 sample.txt
# 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08  sample.txt

openssl dgst -binary -sha256 sample.txt | base64
# n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=

In the browser, assert the one-shot path, the incremental class, and — critically — that chunk boundaries do not change the answer:

const EXPECTED_HEX = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
const EXPECTED_B64 = "n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=";

const sample = new Blob(["test"]);
const oneShot = await digestWholeFile(sample);
console.assert(oneShot.hex === EXPECTED_HEX, "hex mismatch");
console.assert(oneShot.base64 === EXPECTED_B64, "base64 mismatch");

// Two updates across a boundary must equal one update.
const bytes = new TextEncoder().encode("test");
const split = new Sha256().update(bytes.subarray(0, 2)).update(bytes.subarray(2)).digest();
const splitHex = Array.from(split, (b) => b.toString(16).padStart(2, "0")).join("");
console.assert(splitHex === EXPECTED_HEX, "incremental disagrees across a chunk boundary");

// A 1 MB blob hashed in 64 KB slices must equal the one-shot digest.
const big = new Blob([new Uint8Array(1_048_576).fill(7)]);
const worker = new Sha256();
for (let s = 0; s < big.size; s += 65_536) {
  worker.update(new Uint8Array(await big.slice(s, s + 65_536).arrayBuffer()));
}
const chunked = Array.from(worker.digest(), (b) => b.toString(16).padStart(2, "0")).join("");
console.assert(chunked === (await digestWholeFile(big)).hex, "chunked != one-shot");
console.log("all checksum assertions passed");

Then confirm S3 stored and verified it:

aws s3api head-object --bucket uploads --key sample.txt --checksum-mode ENABLED
# "ChecksumSHA256": "n4bQgYhMfWWaL+qgxVrQFaO/TxsrC4Is0V1sFbDwCgg=",
# "ChecksumType": "FULL_OBJECT"

A ChecksumSHA256 ending in -4 instead means the object was written as four multipart parts and the value is a composite root — compare it against digestByParts(...).root, not against shasum.

Frequently Asked Questions