Presigning S3 Multipart Upload Parts #

Call CreateMultipartUpload on the server with the key, content type and any checksum algorithm, store the returned UploadId against the user’s upload record, and expose an endpoint that returns presigned UploadPart URLs for a requested batch of part numbers — each signed for that exact key, upload ID and part number with a short expiry — so the browser can PUT parts straight to S3 and read each part’s ETag for the final CompleteMultipartUpload.

Presigning a single PutObject is the familiar case: one URL, one request. Multipart turns one upload into hundreds of requests, each needing its own signature, and moves three pieces of state — the upload ID, the part numbers, the part ETags — into the conversation between browser and server. Getting the signing endpoint right is what makes large browser uploads both fast (no server in the byte path) and safe (a URL can only write the part it was issued for). This page belongs to S3 multipart upload orchestration in backend validation and cloud storage architecture. The single-object version is generating secure presigned URLs with AWS SDK v3; the browser side is uploading multi-gigabyte files from the browser.

When to use this approach #

• Files are large enough that a single PUT is fragile — over roughly 100 MB, and certainly over S3’s 5 GB single-request limit.
• Browsers upload directly to S3, and you want parallel, retryable parts without a proxy.
• You need per-user authorisation for every part, not a long-lived credential in the browser.

Prerequisites #

1. @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner v3.
2. An IAM role for the signing service with s3:PutObject and s3:AbortMultipartUpload on the upload prefix (UploadPart and CompleteMultipartUpload are authorised by s3:PutObject), and s3:ListMultipartUploadParts for resuming.
3. Bucket CORS allowing PUT from your origin and exposing ETag.
4. A table mapping your upload IDs to S3’s UploadId, key and owner.

Who holds what #

Implementation #

import {
  S3Client, CreateMultipartUploadCommand, UploadPartCommand,
  type ChecksumAlgorithm,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import pg from "pg";
import { randomUUID } from "node:crypto";

const s3 = new S3Client({});
const db = new pg.Pool({ connectionString: process.env.DATABASE_URL });
const BUCKET = process.env.UPLOAD_BUCKET!;
const MAX_FILE = 50 * 1024 ** 3;                 // 50 GB product limit
const MIN_PART = 5 * 1024 ** 2, MAX_PART = 5 * 1024 ** 3, MAX_PARTS = 10_000;
const PART_URL_TTL = 900;                        // seconds; sign in batches so this is plenty
const MAX_BATCH = 50;

/* CREATE TABLE multipart_uploads (
     id uuid PRIMARY KEY, owner_id text NOT NULL, bucket text NOT NULL, key text NOT NULL,
     s3_upload_id text NOT NULL, size bigint NOT NULL, part_size bigint NOT NULL,
     content_type text NOT NULL, status text NOT NULL DEFAULT 'open', created_at timestamptz DEFAULT now()); */

export async function createUpload(ownerId: string, size: number, contentType: string, partSize: number) {
  if (size <= 0 || size > MAX_FILE) throw new Error("size out of range");
  if (partSize < MIN_PART || partSize > MAX_PART) throw new Error("part size out of range");
  if (Math.ceil(size / partSize) > MAX_PARTS) throw new Error("too many parts: increase partSize");

  const id = randomUUID();
  const key = `uploads/${ownerId}/${id}/source`;
  const out = await s3.send(new CreateMultipartUploadCommand({
    Bucket: BUCKET,
    Key: key,
    ContentType: contentType,
    ChecksumAlgorithm: "CRC32C" as ChecksumAlgorithm,   // parts must then carry a CRC32C
    Metadata: { "owner-id": ownerId, "upload-id": id },
  }));
  await db.query(
    `INSERT INTO multipart_uploads (id, owner_id, bucket, key, s3_upload_id, size, part_size, content_type)
     VALUES ($1,$2,$3,$4,$5,$6,$7,$8)`,
    [id, ownerId, BUCKET, key, out.UploadId, size, partSize, contentType]);
  return { id, partSize, parts: Math.ceil(size / partSize) };
}

export async function signParts(
  ownerId: string, id: string, partNumbers: number[], checksums?: Record<number, string>,
): Promise<Record<number, string>> {
  const { rows } = await db.query(
    `SELECT key, s3_upload_id, size, part_size FROM multipart_uploads
      WHERE id = $1 AND owner_id = $2 AND status = 'open'`, [id, ownerId]);
  const u = rows[0];
  if (!u) throw new Error("upload not found or not open");
  const total = Math.ceil(Number(u.size) / Number(u.part_size));
  const wanted = [...new Set(partNumbers)].filter((n) => Number.isInteger(n) && n >= 1 && n <= total).slice(0, MAX_BATCH);

  const urls: Record<number, string> = {};
  await Promise.all(wanted.map(async (n) => {
    const len = n < total ? Number(u.part_size) : Number(u.size) - (total - 1) * Number(u.part_size);
    urls[n] = await getSignedUrl(s3, new UploadPartCommand({
      Bucket: BUCKET,
      Key: u.key,
      UploadId: u.s3_upload_id,
      PartNumber: n,
      ContentLength: len,                                   // exact bytes this part may contain
      ...(checksums?.[n] ? { ChecksumCRC32C: checksums[n] } : {}),
    }), { expiresIn: PART_URL_TTL, signableHeaders: new Set(["content-length"]) });
  }));
  return urls;
}

// Usage from route handlers
const { id } = await createUpload("user-42", 21_474_836_480, "video/quicktime", 64 * 1024 ** 2);
console.log(Object.keys(await signParts("user-42", id, [1, 2, 3, 4, 5])).length);
// 5

Line-by-line on the parameters that matter #

• Validating size and part count before CreateMultipartUpload. S3 allows at most 10,000 parts and parts between 5 MiB and 5 GiB (the last part may be smaller). Checking up front turns a failure at part 10,001 — hours into an upload — into an immediate error.
• Server-generated key with the owner in it. The client never chooses where its bytes go. Including the owner and your own upload ID in the key makes authorisation checks and clean-up trivial, and is the basis for the prefix-scoped policies in scoping upload keys per user with IAM policy variables.
• ChecksumAlgorithm: "CRC32C" at creation. Declaring the algorithm on the multipart upload means every part must carry that checksum and S3 computes a composite checksum for the object — the integrity story in verifying uploads with S3 additional checksums. If you do not want per-part hashing in the browser, omit it here and in signParts.
• Authorisation on every batch. signParts looks up the upload by your ID and the caller’s identity. A client that learns someone else’s upload ID still cannot obtain URLs for it.
• ContentLength signed per part. Each URL accepts exactly the number of bytes that part should contain. A client cannot upload an oversized part to exceed quotas.
• Batches of at most 50, 15-minute expiry. The browser asks for URLs a few batches ahead of need. Short-lived URLs limit what a leaked URL can do; batching avoids a signing round trip per part.

Signing cost and batch size #

Security properties of a part URL #

A presigned UploadPart URL is narrower than it looks. The signature covers the HTTP method, bucket, key, the uploadId and partNumber query parameters, the expiry, and any signed headers such as Content-Length and the part checksum. Changing any of them invalidates it. So a leaked part URL lets the holder do exactly one thing: upload bytes of that exact length into that one part of that one in-progress upload, until it expires — and the owner’s later completion will either include those bytes (if the attacker’s part replaced theirs) or fail because the ETag no longer matches.

That residual risk is why the checksum matters beyond integrity. With ChecksumCRC32C signed into the URL, S3 rejects any body whose checksum differs from the one the owner’s browser computed, which means a leaked URL cannot be used to substitute different content at all. Combined with short expiries and HTTPS-only access (a bucket policy denying aws:SecureTransport = false), part URLs are safe to hand to a browser.

What a part URL cannot enforce is anything about the whole object: total size, final content type, what the file actually is. Those checks belong to completion — completing and aborting S3 multipart uploads — and to post-upload validation.

Configuration gotchas #

SignatureDoesNotMatch on every part. The browser adds headers you did not sign (Content-Type on a part, which UploadPart does not use), or the signed ContentLength differs from the actual slice. Send only the body and the signed headers; compute the last part’s length exactly.

ETag is null in the browser. CORS does not expose it. Add ETag to the bucket CORS ExposeHeaders; without it the browser cannot complete the upload.

NoSuchUpload when signing or uploading. The multipart upload was aborted — by your code or by a lifecycle rule that expires incomplete uploads. Mark the record closed and start a new upload.

InvalidRequest: Checksum Type mismatch. The upload was created with ChecksumAlgorithm: CRC32C but a part was sent with a SHA-256 or no checksum. Keep the algorithm consistent across create, sign and part requests.

The request sequence #

Verification #

# Create and sign via your API, then upload part 1 by hand.
URL=$(curl -s -X POST localhost:8080/api/multipart/$ID/sign -H "Authorization: Bearer $T" \
  -d '{"parts":[1]}' -H 'Content-Type: application/json' | jq -r '.["1"]')
head -c 67108864 big.bin | curl -s -D - -o /dev/null -X PUT --data-binary @- "$URL" | grep -i etag
# etag: "9b2cf535f27731c974343645a3985328"

# The part is visible to ListParts but not to ListObjects.
aws s3api list-parts --bucket "$BUCKET" --key "$KEY" --upload-id "$S3_UPLOAD_ID" --query 'Parts[].[PartNumber,Size]'

# A wrong-length body is refused.
head -c 1000 big.bin | curl -s -o /dev/null -w '%{http_code}\n' -X PUT --data-binary @- "$URL"
# 403

Frequently Asked Questions