Verifying Uploads with S3 Additional Checksums #

Compute a CRC32C (fast) or SHA-256 (cryptographic) for each part in the browser, send it with the part as x-amz-checksum-crc32c so S3 rejects any part whose bytes differ, create the multipart upload with ChecksumType: "FULL_OBJECT" for CRC algorithms when you want one checksum for the whole file, and after completion read it back with HeadObject(ChecksumMode: "ENABLED") to compare against the value your client computed over the entire file.

An upload that “succeeded” can still be wrong: a proxy that re-chunks bodies, a buggy slice at a part boundary, a memory error on a phone, a retry that sent a different part’s bytes. The traditional check — comparing an MD5 against the ETag — stops working for multipart uploads, whose ETag is an MD5 of MD5s. S3’s additional checksums (CRC32, CRC32C, CRC64NVME, SHA-1, SHA-256) give you integrity per part and for the whole object, verified by S3 on receipt. This page belongs to S3 multipart upload orchestration in backend validation and cloud storage architecture. For hashing in the browser, see computing file checksums in the browser with Web Crypto.

When to use this approach #

• Uploads are large or valuable enough that silent corruption would be expensive — video masters, datasets, legal documents, backups.
• You need proof later that the stored object is exactly what the user sent.
• You want to deduplicate or verify files against a hash the client computed before upload.

Prerequisites #

1. @aws-sdk/client-s3 3.700 or newer (full-object checksums for multipart and default integrity protections).
2. A browser CRC32C or SHA-256 implementation: Web Crypto for SHA-256 (per part, since digest() is one-shot), or a small WebAssembly library such as hash-wasm for CRC32C and streaming SHA-256.
3. Presigned part URLs that sign the checksum header — see presigning S3 multipart upload parts.
4. CORS that allows the x-amz-checksum-* request header.

Composite versus full-object checksums #

Implementation #

Browser side: compute a CRC32C per part while also accumulating a whole-file CRC32C, and send the part value with each upload.

import { createCRC32C } from "hash-wasm";

const b64 = (hex: string) => btoa(String.fromCharCode(...hex.match(/../g)!.map((h) => parseInt(h, 16))));

export async function partChecksums(file: File, partSize: number): Promise<{ parts: string[]; full: string }> {
  const whole = await createCRC32C();
  whole.init();
  const parts: string[] = [];
  for (let off = 0; off < file.size; off += partSize) {
    const part = await createCRC32C();
    part.init();
    const reader = file.slice(off, Math.min(off + partSize, file.size)).stream().getReader();
    for (;;) {
      const { done, value } = await reader.read();
      if (done) break;
      part.update(value);
      whole.update(value);
    }
    parts.push(b64(part.digest("hex")));               // S3 wants base64 of the big-endian bytes
  }
  return { parts, full: b64(whole.digest("hex")) };
}

export async function putPart(url: string, body: Blob, crc32cB64: string): Promise<string> {
  const res = await fetch(url, { method: "PUT", body, headers: { "x-amz-checksum-crc32c": crc32cB64 } });
  if (res.status === 400) throw new Error(`S3 rejected part: ${await res.text()}`);   // BadDigest
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  return res.headers.get("ETag")!;
}

Server side: create with a full-object CRC32C, sign parts with their checksums, complete with them, and verify the final object against the client’s whole-file value.

import {
  S3Client, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, HeadObjectCommand,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const s3 = new S3Client({});
const Bucket = process.env.UPLOAD_BUCKET!;

export async function create(Key: string) {
  const out = await s3.send(new CreateMultipartUploadCommand({
    Bucket, Key, ChecksumAlgorithm: "CRC32C", ChecksumType: "FULL_OBJECT",
  }));
  return out.UploadId!;
}

export function signPart(Key: string, UploadId: string, PartNumber: number, len: number, crc: string) {
  return getSignedUrl(s3, new UploadPartCommand({
    Bucket, Key, UploadId, PartNumber, ContentLength: len, ChecksumCRC32C: crc,
  }), { expiresIn: 900, signableHeaders: new Set(["content-length", "x-amz-checksum-crc32c"]) });
}

export async function completeAndVerify(
  Key: string, UploadId: string,
  parts: { PartNumber: number; ETag: string; ChecksumCRC32C: string }[],
  clientFullCrc: string,
): Promise<{ ok: boolean; stored?: string }> {
  await s3.send(new CompleteMultipartUploadCommand({
    Bucket, Key, UploadId,
    ChecksumType: "FULL_OBJECT",
    ChecksumCRC32C: clientFullCrc,            // S3 rejects completion if its computed value differs
    MultipartUpload: { Parts: parts },
  }));
  const head = await s3.send(new HeadObjectCommand({ Bucket, Key, ChecksumMode: "ENABLED" }));
  return { ok: head.ChecksumCRC32C === clientFullCrc, stored: head.ChecksumCRC32C };
}

Line-by-line on the details that matter #

• Signing the checksum header. Including x-amz-checksum-crc32c in the signed headers means the URL only accepts a body with that exact checksum. S3 recomputes it on receipt and returns 400 BadDigest if the bytes differ — corruption in transit is rejected before it is stored.
• Base64 of the raw digest. S3 expects the checksum as base64 of the big-endian binary value, not hex. Sending hex produces InvalidRequest: Value for x-amz-checksum-crc32c header is invalid.
• ChecksumType: "FULL_OBJECT". For CRC algorithms, S3 combines part CRCs mathematically into the CRC of the whole byte stream, so the result equals a CRC32C computed over the file in one pass, whatever the part size. SHA algorithms cannot be combined this way and only support composite checksums.
• ChecksumCRC32C on completion. Passing the whole-file value makes S3 verify it during completion; a mismatch fails the completion instead of producing an object with a checksum nobody checks.
• Hashing while slicing. The part and the whole-file hash are updated from the same chunks, so the file is read once for hashing. For multi-gigabyte files, do this in a worker; CRC32C in WebAssembly runs at several hundred megabytes per second.

Cost of hashing in the browser #

Choosing an algorithm #

Pick by what the checksum is for. Integrity in transit and at rest — detecting accidental corruption — is exactly what CRCs are designed for, and CRC32C has hardware support in most CPUs and a full-object mode. Identity — deduplication, proving to a third party which file was stored, matching against a known hash — needs a cryptographic hash, because a CRC can be forged trivially. SHA-256 serves that purpose, but S3 can only store it as a composite for multipart uploads, which does not match a whole-file SHA-256 computed elsewhere.

A common arrangement uses both: CRC32C with FULL_OBJECT for S3’s own verification of every upload, and a whole-file SHA-256 computed in the browser (or by a worker after upload) stored in your database for deduplication and audit — the approach in deduplicating uploads with content hashes. The two answer different questions and neither replaces the other.

Keeping checksums useful after the upload #

A checksum that is only checked once, at upload time, protects against transit errors. A checksum that is stored and consulted later protects against much more: a bug in your processing that overwrites the original, a botched migration between buckets, a restore from backup that picked up the wrong version. Three habits keep that value.

Store the whole-file value in your database. Record the client-computed checksum with the asset at confirmation time, next to the size. Every later operation that copies, moves or restores the object can compare against it without trusting the storage layer’s own metadata.

Carry checksums across copies. CopyObject and S3 Batch Operations can compute a new checksum during the copy when you pass ChecksumAlgorithm. Compare it with the stored value after any bulk copy or cross-region replication job — a mismatch is a real defect, found before anyone downloads a broken file.

Verify on read for critical files. For archival or legal content, fetch with ChecksumMode: "ENABLED" and let the SDK validate the response body against the stored checksum as it streams. The cost is a little CPU on the reader; the benefit is that corrupted bytes are never served silently.

Configuration gotchas #

BadDigest: The CRC32C you specified did not match the calculated checksum. The body differs from what was hashed — a wrong slice boundary, or hashing hex text instead of bytes. Hash exactly the Blob you send.

Browser request fails CORS preflight after adding the header. x-amz-checksum-crc32c is a non-safelisted header; the bucket CORS configuration must list it (or *) in AllowedHeaders.

HeadObject returns no checksum. You did not pass ChecksumMode: "ENABLED", or the object was uploaded without additional checksums. Objects written before you enabled checksums have none; CopyObject onto itself with ChecksumAlgorithm computes one.

Composite checksum does not match your whole-file SHA-256. Expected: composite values are hashes of part hashes with a -N suffix. Compare whole-file hashes only with FULL_OBJECT CRCs, or keep your own SHA-256.

Verification #

# Per-part checksum enforced: a corrupted body is rejected.
printf 'x' | cat - part1.bin | head -c "$(stat -c %s part1.bin)" > bad.bin
curl -s -o /dev/null -w '%{http_code}\n' -X PUT -H "x-amz-checksum-crc32c: $CRC1" --data-binary @bad.bin "$PART1_URL"
# 400

# Whole-object checksum stored and matching the client's.
aws s3api head-object --bucket "$BUCKET" --key "$KEY" --checksum-mode ENABLED \
  --query '[ChecksumCRC32C, ChecksumType]'
# [ "yZRlqg==", "FULL_OBJECT" ]

Frequently Asked Questions