Post-Upload Media Transcoding #

A successful upload is not a finished job. What landed in the bucket is a 180 MB ProRes clip or a 24-megapixel HEIC straight off an iPhone; what your app has to serve is a 40 KB avatar, a poster frame and an H.264 rendition that plays on a five-year-old Android — and the conversion between the two is the part that takes seconds to minutes of CPU, fails in interesting ways, and has to be re-runnable when someone changes the thumbnail size next quarter.

This topic sits inside backend validation and cloud storage architecture. It picks up where the byte-level work ends: the object is durable, server-side file validation has confirmed it is what it claims to be, and a scanner has cleared it. Everything below is about the asynchronous machine that turns that original into a derivative set, and about the four or five ways that machine goes wrong in production.

Prerequisites #

• [ ] Node 20 or newer — the worker code uses node:stream/promises, AbortSignal.timeout() and top-level await.
• [ ] An S3 bucket with either event notifications or EventBridge enabled, plus write access to a second prefix or bucket for outputs.
• [ ] ffmpeg 6.x and ffprobe on PATH (a static build, a Lambda layer, or a container base image), and sharp 0.33+ installed for the platform you deploy to, not the platform you develop on.
• [ ] An SQS standard queue with a redrive policy pointing at a dead-letter queue.
• [ ] A transactional store for job state — Postgres or DynamoDB — with a unique key you can write conditionally.
• [ ] IAM for the worker: s3:GetObject on the originals prefix, s3:PutObject on the derivatives prefix, and sqs:ReceiveMessage, sqs:DeleteMessage, sqs:ChangeMessageVisibility on the queue.

How it works #

The mechanism is four hops, and every one of them has a delivery guarantee you need to know.

The upload completes and S3 emits an event. Which event matters: a single PUT produces s3:ObjectCreated:Put, while a multipart upload produces s3:ObjectCreated:CompleteMultipartUpload and nothing at all for the individual parts. If you subscribe only to Put — an easy mistake, because that is what the console suggests first — every file large enough to trigger the multipart path in direct-to-cloud upload patterns silently never transcodes. Subscribe to s3:ObjectCreated:* unless you have a specific reason not to.

The event goes to a destination. S3 can push straight to SQS, SNS or Lambda, or you can enable EventBridge on the bucket and route from there. Direct-to-SQS is one hop fewer, but S3’s native notification configuration rejects overlapping rules: two rules for the same event type whose prefixes overlap fail the PutBucketNotificationConfiguration call with Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type. The moment you want a second consumer, you are rewriting the configuration. EventBridge costs about a millisecond of latency and $1 per million events, and lets any number of rules match the same object independently.

Delivery is at-least-once and unordered. AWS documents notifications as “typically delivered in seconds” but explicitly allows a minute or more, and duplicates are normal — not rare enough to ignore. Overwriting the same key twice in quick succession produces two events that can arrive in either order; the sequencer field in the event record is a hex string you can compare lexicographically to tell which write came later, and it is the only ordering signal you get.

The worker pulls from the queue, downloads the original, produces derivatives, writes them to a different prefix, and updates a row. It deletes the message only after that row is committed.

Why this never goes in the request path #

The numbers make the argument on their own. Resizing a 24 MP JPEG into three variants with Sharp takes 700–900 ms of wall time on one vCPU and peaks around 180 MB of resident memory. Transcoding sixty seconds of 1080p H.264 to a 720p rendition with libx264 -preset veryfast takes 8–15 seconds on two vCPUs, and a ten-minute clip takes ten to twenty minutes. Meanwhile API Gateway has a hard 29-second integration timeout you cannot raise, ALB defaults to a 60-second idle timeout, and CloudFront caps origin response time at 60 seconds by default.

Even where the request would fit, the shape is wrong. Transcoding in the handler means the upload’s HTTP connection is the transaction boundary: a user closing the tab kills the work, a deploy kills the work, and there is no retry except asking the user to upload again. Twenty concurrent uploads become twenty FFmpeg processes on an instance sized for JSON. Push the work onto a queue and all of that becomes configuration — concurrency is a queue setting, retries are a redrive policy, and a deploy just means messages sit for thirty seconds.

The user-facing cost is that the UI must now model “uploaded but not ready”. That is a feature, not a regression: it is the same state machine you already need for scanning, and you can surface it honestly with streaming upload progress with server-sent events rather than pretending the spinner ended when the bytes did.

Bucket and prefix layout #

Get this wrong and you will discover it via the bill.

Originals and derivatives must not share a watched prefix. If the worker writes derivatives/abc/thumb.webp into the same prefix the trigger matches, that write raises another ObjectCreated event, which enqueues another job, which writes another file. AWS will happily run that loop until you notice. Either use two buckets, or one bucket with two top-level prefixes and a trigger scoped to originals/ only — and then verify the scoping, because a prefix filter of "" matches everything.

The layout that has held up for me:

uploads-prod/
  originals/2026/07/26/9f3c1b2a/source.mov     ← trigger prefix, read-only to workers
  derivatives/9f3c1b2a/v3/poster_1280.jpg      ← never watched, immutable, CDN-cacheable
  derivatives/9f3c1b2a/v3/720p.mp4
  derivatives/9f3c1b2a/v3/manifest.json

Three things that layout buys you. Derivative URLs are computable — given a media id and the current recipe version, the frontend builds the URL without a round trip. A recipe change bumps v3 to v4, which is a new path, which means Cache-Control: public, max-age=31536000, immutable is safe and you never issue a CDN invalidation. And the two prefixes get different retention: originals are expensive and rarely re-read, so they belong in Glacier Instant Retrieval after thirty days, while v2/ derivatives can be deleted outright a week after v3/ ships. Both are one rule each in cloud storage lifecycle rules.

Date-partition the originals prefix (2026/07/26/) and not the derivatives. S3 scales request rate per partitioned prefix — 3,500 writes and 5,500 reads per second — and a backfill that re-reads every original in date order spreads across partitions naturally. Derivatives are read by media id, which is already high-cardinality.

Idempotent job keys #

At-least-once delivery means your worker will process the same object twice. Usually that is harmless — the second run writes the same bytes to the same key — but “usually” hides two real problems: you pay twice for the CPU, and a job that appends (an analytics row, a webhook, a notification email) does it twice.

The fix is a deterministic job key and a conditional write, not a distributed lock. Derive the key from everything that determines the output:

import { createHash } from "node:crypto";

export interface Recipe {
  id: string;
  version: number;
}

export interface SourceRef {
  bucket: string;
  key: string;
  versionId: string; // "null" on an unversioned bucket — use the ETag instead
}

export function jobKey(src: SourceRef, recipe: Recipe): string {
  return createHash("sha256")
    .update([src.bucket, src.key, src.versionId, recipe.id, String(recipe.version)].join("�"))
    .digest("hex")
    .slice(0, 32);
}

The � separator matters: joining with : lets a key containing a colon collide with a different bucket-and-key pair. The versionId matters because overwriting an original must produce a different job — with an unversioned bucket, substitute the ETag from the event record, which for a single PUT is the MD5 of the body.

Claim the job with a conditional insert. In Postgres a single statement does claim, dedupe and retry accounting:

import pg from "pg";

const pool = new pg.Pool({ max: 4 });

export type Claim = "claimed" | "duplicate";

export async function claimJob(key: string, src: SourceRef, recipe: Recipe): Promise<Claim> {
  const { rowCount } = await pool.query(
    `INSERT INTO media_jobs (job_key, source_key, source_version, recipe_id, recipe_version, state)
     VALUES ($1, $2, $3, $4, $5, 'running')
     ON CONFLICT (job_key) DO UPDATE
       SET state = 'running', attempts = media_jobs.attempts + 1, started_at = now()
       WHERE media_jobs.state = 'transient_failure'
     RETURNING job_key`,
    [key, src.key, src.versionId, recipe.id, recipe.version],
  );
  return rowCount === 1 ? "claimed" : "duplicate";
}

A fresh event inserts and returns a row. A duplicate event for a job that is running, ready or permanent_failure hits the conflict, fails the WHERE, updates nothing and returns rowCount === 0 — the worker deletes the message and does no work. Only a job previously marked transient_failure is re-claimed. This is the same idempotency-token discipline the client side uses in retrying fetch uploads with idempotency keys, applied one layer down.

For the derivative writes themselves, S3 conditional writes remove the last race. PutObjectCommand accepts IfNoneMatch: "*", which fails with 412 PreconditionFailed if the key already exists — so two workers that somehow both claimed the job cannot half-overwrite each other’s output. Treat the 412 as success.

Step-by-step implementation #

1. Route events into a queue with a redrive policy #

Enable EventBridge on the bucket, then point a rule at the queue. The queue policy must restrict the source, or anyone who guesses the URL can inject jobs:

aws s3api put-bucket-notification-configuration \
  --bucket uploads-prod \
  --notification-configuration '{"EventBridgeConfiguration":{}}'

aws events put-rule --name transcode-on-upload \
  --event-pattern '{
    "source": ["aws.s3"],
    "detail-type": ["Object Created"],
    "detail": { "bucket": { "name": ["uploads-prod"] },
                "object": { "key": [{ "prefix": "originals/" }] } }
  }'

aws sqs set-queue-attributes --queue-url "$QUEUE_URL" --attributes '{
  "VisibilityTimeout": "960",
  "MessageRetentionPeriod": "1209600",
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:eu-west-1:111122223333:transcode-dlq\",\"maxReceiveCount\":\"5\"}"
}'

VisibilityTimeout of 960 seconds is six times a 160-second worker budget — AWS’s own guidance for SQS-triggered Lambda, and a sane floor for containers too. MessageRetentionPeriod is the maximum fourteen days, so a queue that backs up over a long weekend loses nothing.

Verify the rule matches before you trust it:

aws events test-event-pattern \
  --event-pattern file://pattern.json \
  --event '{"source":"aws.s3","detail-type":"Object Created","account":"111122223333",
            "region":"eu-west-1","time":"2026-07-26T09:00:00Z","resources":[],
            "detail":{"bucket":{"name":"uploads-prod"},
                      "object":{"key":"originals/2026/07/26/9f3c1b2a/source.mov"}}}'
# { "Result": true }

2. Define the recipe as data, not code #

A recipe that lives in a config object can be versioned, hashed, diffed in review and backfilled. One buried in a function body cannot.

export const VIDEO_RECIPE = {
  id: "web-video",
  version: 3,
  poster: { atPercent: 10, width: 1280 },
  renditions: [
    { name: "720p", height: 720, crf: 23, maxrate: "2500k", bufsize: "5000k", audioKbps: 128 },
    { name: "360p", height: 360, crf: 26, maxrate: "800k", bufsize: "1600k", audioKbps: 96 },
  ],
} as const;

export const IMAGE_RECIPE = {
  id: "web-image",
  version: 3,
  variants: [
    { name: "thumb", width: 320, format: "webp", quality: 72 },
    { name: "card", width: 800, format: "webp", quality: 78 },
    { name: "full", width: 1600, format: "avif", quality: 50 },
  ],
} as const;

Bumping version invalidates every job key derived from it, so replaying the originals through the queue regenerates everything into a fresh v4/ prefix while v3/ keeps serving traffic.

3. Run FFmpeg as a child process, not through a wrapper #

Spawn the binary with an argv array. Never build a shell string — a filename containing ; becomes remote code execution, and the same magic-byte scepticism from detecting file type from magic bytes in JavaScript applies to anything derived from a user-supplied name.

import { spawn } from "node:child_process";
import { once } from "node:events";

export class TranscodeError extends Error {
  constructor(
    message: string,
    readonly exitCode: number | null,
    readonly permanent: boolean,
  ) {
    super(message);
    this.name = "TranscodeError";
  }
}

// stderr patterns that no amount of retrying will fix
const PERMANENT = [
  /moov atom not found/i,
  /Invalid data found when processing input/i,
  /does not contain any stream/i,
  /Unknown encoder/i,
];

export async function runFfmpeg(
  args: string[],
  onProgressMs: (outMs: number) => void,
  timeoutMs = 840_000,
): Promise<void> {
  const child = spawn(
    "ffmpeg",
    ["-nostdin", "-hide_banner", "-loglevel", "error", "-progress", "pipe:1", "-y", ...args],
    { stdio: ["ignore", "pipe", "pipe"] },
  );

  const killer = setTimeout(() => child.kill("SIGKILL"), timeoutMs);
  let stderr = "";
  child.stderr.setEncoding("utf8");
  child.stderr.on("data", (d: string) => {
    stderr = (stderr + d).slice(-8192); // keep the tail, never the whole log
  });

  let buf = "";
  child.stdout.setEncoding("utf8");
  child.stdout.on("data", (d: string) => {
    buf += d;
    let nl = buf.indexOf("\n");
    while (nl !== -1) {
      const [k, v] = buf.slice(0, nl).split("=");
      buf = buf.slice(nl + 1);
      // out_time_ms is documented in milliseconds and emitted in microseconds
      if (k === "out_time_ms") onProgressMs(Number(v) / 1000);
      nl = buf.indexOf("\n");
    }
  });

  const [code, signal] = (await once(child, "close")) as [number | null, string | null];
  clearTimeout(killer);
  if (code === 0) return;

  const tail = stderr.trim().split("\n").at(-1) ?? "no stderr output";
  throw new TranscodeError(
    `ffmpeg exited ${code ?? signal}: ${tail}`,
    code,
    PERMANENT.some((re) => re.test(stderr)),
  );
}

Two details earn their place. -nostdin stops FFmpeg consuming the parent’s stdin and hanging forever when it decides to ask “File exists. Overwrite?”. And out_time_ms is a long-standing FFmpeg wart: the key says milliseconds, the value is microseconds. Divide by 1,000 or your progress bar reports a ten-second clip as nearly three hours.

Building the 720p rendition:

import { join } from "node:path";

export function renditionArgs(input: string, outDir: string, r: typeof VIDEO_RECIPE.renditions[number]) {
  return [
    "-i", input,
    "-vf", `scale=-2:${r.height}`,        // -2 keeps width even; libx264 rejects odd widths
    "-c:v", "libx264", "-preset", "veryfast", "-crf", String(r.crf),
    "-maxrate", r.maxrate, "-bufsize", r.bufsize,
    "-profile:v", "high", "-level", "4.0", "-pix_fmt", "yuv420p",
    "-movflags", "+faststart",            // moov atom to the front, or Safari waits for the last byte
    "-c:a", "aac", "-b:a", `${r.audioKbps}k`, "-ac", "2",
    join(outDir, `${r.name}.mp4`),
  ];
}

Expected worker log for a 62-second 1080p source on two vCPUs:

[job 41c8…] claimed  src=originals/2026/07/26/9f3c1b2a/source.mov 184.2 MB
[job 41c8…] probe    1920x1080 h264 62.04s 30000/1001 fps
[job 41c8…] 720p     progress 12.0s/62.0s (19%)
[job 41c8…] 720p     progress 48.5s/62.0s (78%)
[job 41c8…] 720p     done in 11.4s -> 18.9 MB
[job 41c8…] 360p     done in 5.1s -> 6.2 MB
[job 41c8…] poster   done in 0.4s -> 214 KB
[job 41c8…] uploaded 3 derivatives, state=ready, total 19.7s

4. Generate image derivatives with Sharp #

Sharp is libvips underneath, which streams rather than decoding the whole image into memory, and it is roughly four times faster than an ImageMagick shell-out for the same work. The subtlety is that a sharp instance is single-use for output: call .clone() for each variant or the second .toBuffer() throws.

import sharp from "sharp";

export interface Variant {
  name: string;
  ext: "webp" | "avif";
  body: Buffer;
  width: number;
  height: number;
}

export async function renderImageVariants(input: Buffer): Promise<Variant[]> {
  const base = sharp(input, {
    limitInputPixels: 50_000_000, // ~50 MP; the default 268 MP is a decompression-bomb budget
    failOn: "truncated",          // reject a half-written upload, tolerate benign warnings
    sequentialRead: true,
  })
    .rotate()                     // apply EXIF orientation BEFORE resize, or portrait shots come out sideways
    .withMetadata({ icc: "srgb" }); // keep a colour profile or wide-gamut phone photos wash out

  const out: Variant[] = [];
  for (const v of IMAGE_RECIPE.variants) {
    const pipeline = base.clone().resize({ width: v.width, withoutEnlargement: true });
    const { data, info } =
      v.format === "avif"
        ? await pipeline.avif({ quality: v.quality, effort: 4 }).toBuffer({ resolveWithObject: true })
        : await pipeline.webp({ quality: v.quality, effort: 4 }).toBuffer({ resolveWithObject: true });
    out.push({ name: v.name, ext: v.format, body: data, width: info.width, height: info.height });
  }
  return out;
}

withoutEnlargement: true is the difference between a 320-pixel avatar and a 320-pixel upscale of a 96-pixel source that looks worse than the original. effort: 4 on AVIF is the honest middle: effort: 9 buys about 8% more compression for roughly six times the CPU, which at scale is money you are spending on encoder time to save on transfer.

5. Keep the message alive while the job runs #

A job longer than the visibility timeout gets redelivered while the first worker is still encoding — two workers, same file, doubled cost, and eventually a DLQ entry for a job that actually succeeded. Extend the lease on a timer:

import { SQSClient, ChangeMessageVisibilityCommand } from "@aws-sdk/client-sqs";

export function startHeartbeat(
  sqs: SQSClient,
  queueUrl: string,
  receiptHandle: string,
  extendBy = 300,
  everyMs = 120_000,
): () => void {
  const timer = setInterval(() => {
    sqs
      .send(new ChangeMessageVisibilityCommand({
        QueueUrl: queueUrl,
        ReceiptHandle: receiptHandle,
        VisibilityTimeout: extendBy,
      }))
      .catch((err: Error) => {
        // ReceiptHandleIsInvalid means the message already moved on — stop encoding.
        console.warn(`heartbeat failed: ${err.name}`);
      });
  }, everyMs);
  timer.unref();
  return () => clearInterval(timer);
}

Log a ReceiptHandleIsInvalid loudly and abort the encode: the message has already been redelivered or dead-lettered, so finishing the work only wastes CPU.

6. Commit the manifest last #

Write every derivative, then one manifest.json, then the database row. The manifest is the commit marker — a reader that sees it knows the set is complete, and a job that died after two of three uploads leaves no manifest and gets cleanly redone. Store the intrinsic facts the app needs alongside it, following the conventions in storing image dimensions and duration metadata:

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({});

export async function putDerivative(bucket: string, key: string, body: Buffer, contentType: string) {
  try {
    await s3.send(new PutObjectCommand({
      Bucket: bucket,
      Key: key,
      Body: body,
      ContentType: contentType,
      CacheControl: "public, max-age=31536000, immutable",
      IfNoneMatch: "*", // 412 if a concurrent worker already wrote it
    }));
    return "written";
  } catch (err) {
    if ((err as { name?: string }).name === "PreconditionFailed") return "already-present";
    throw err;
  }
}

Choosing the compute #

Lambda and a container fleet are both correct answers; the split is not really about cost.

Lambda gives you 128 MB to 10,240 MB of memory, with vCPU scaling linearly — one full vCPU at 1,769 MB and about six at the ceiling — a hard 15-minute timeout, and /tmp configurable from 512 MB to 10,240 MB. The 250 MB unzipped package limit rules out bundling FFmpeg as a zip, so ship a container image (up to 10 GB) or a layer. Cold starts for a 400 MB FFmpeg image run 2–6 seconds, which is invisible behind a queue and unacceptable in a request path.

A Fargate or EC2 service has no timeout, keeps FFmpeg’s thread pool warm across jobs, and can hold a 40 GB scratch volume. It also costs money while idle and needs autoscaling wired to ApproximateNumberOfMessagesVisible.

The arithmetic behind that chart: an 800 ms job at 2,048 MB on arm64 costs 1.64 GB-seconds, or about $0.0000221 including the request charge. Ten thousand images a day is $6.60 a month — genuinely free. A single 1 vCPU / 2 GB Fargate task is $35.55 a month whether it works or not, and at 70% average utilisation absorbs roughly 75,000 of those jobs a day. Fargate Spot cuts that by about 70% if your jobs are interruptible, which queue-driven transcoding is by construction.

The practical rule: images and poster frames on Lambda, anything that transcodes a full timeline on containers. A ten-minute 1080p source encoding at 1.5× realtime needs about twenty minutes for a two-rung ladder — it cannot fit in Lambda at any memory setting, and splitting it into segments to fit is a much larger project than running a container. That split is exactly how queueing transcode jobs with SQS and Lambda sets up the serverless half.

Failure handling #

Three failure classes, three different responses.

Transient — S3 SlowDown, a 500 from the SDK, a database connection reset, a Spot interruption. Throw and let the message become visible again. The redrive policy gives you five attempts across roughly an hour, which covers every realistic blip.

Permanent — the object is 0 bytes, the MOV has a truncated moov atom, the “image” is an HTML error page saved with a .jpg extension. FFmpeg exits 1 with moov atom not found and will do so on all five attempts, burning five times the CPU and taking an hour to reach the DLQ. Classify it: on a TranscodeError with permanent === true, write state = 'permanent_failure' with the stderr tail, delete the message, and surface it to the user. That is the whole point of the PERMANENT regex list.

Poison-adjacent — the job is legal but pathological. A 200-megapixel PNG, a 14-hour MP4, a file that decodes into 60 GB of raw frames. These fail on resource limits, look transient, and retry forever. Reject them at claim time from the size and probe metadata rather than discovering them at minute twelve, which is the same reasoning behind the quarantine gate in quarantine bucket patterns for infected uploads.

With a Lambda event source, return partial batch failures so one bad record does not re-drive nine good ones:

interface SqsRecord { messageId: string; body: string; receiptHandle: string }
interface SqsEvent { Records: SqsRecord[] }

export async function handler(event: SqsEvent) {
  const batchItemFailures: { itemIdentifier: string }[] = [];
  for (const record of event.Records) {
    try {
      await processOne(JSON.parse(record.body));
    } catch (err) {
      if (err instanceof TranscodeError && err.permanent) {
        console.error(`permanent failure ${record.messageId}: ${err.message}`);
        continue; // deleted with the batch — do not retry
      }
      console.warn(`transient failure ${record.messageId}: ${(err as Error).message}`);
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }
  return { batchItemFailures };
}

This requires FunctionResponseTypes: ["ReportBatchItemFailures"] on the event source mapping. Without it, one failure in a batch of ten redelivers all ten — a spectacular way to multiply cost during an incident.

Alarm on ApproximateNumberOfMessagesVisible on the dead-letter queue at any value above zero, and on ApproximateAgeOfOldestMessage on the main queue above roughly three times your p99 job duration. Redrive with aws sqs start-message-move-task --source-arn "$DLQ_ARN" once the cause is fixed; because the job keys are deterministic, replaying a fixed batch is safe.

Configuration reference #

Edge cases and gotchas #

The recursive trigger loop #

Writing derivatives into the watched prefix re-fires the trigger. The tell is a queue whose depth grows while every job succeeds, and a Lambda invocation graph that goes exponential over about four minutes. Scope the event filter to originals/ and add a defensive guard in the handler — if (key.startsWith("derivatives/")) return; — because a filter is a configuration and configurations get edited.

Zero-byte objects and aborted multipart uploads #

An abandoned multipart upload leaves no object, but a client that PUTs an empty body leaves a real 0-byte one, and the event fires normally. FFmpeg reports Invalid data found when processing input and Sharp throws Input Buffer is empty. Check detail.object.size === 0 in the handler and fail fast — it costs one comparison and saves a download, a decode and four retries. Expiring the abandoned uploads themselves is a lifecycle concern, not a worker one.

The poster frame at t=0 is usually black #

Most video starts on a fade-in or a slate. Seeking to a fixed percentage of the duration gives a usable frame far more often: -ss at 10% of the probed duration, placed before -i so FFmpeg seeks by keyframe index instead of decoding from the start — the difference between 0.4 seconds and 40 on a long file. FFmpeg’s thumbnail filter picks the most representative frame from a window, at the cost of decoding that window. Both approaches, and the variable-frame-rate trap where -vf fps=1 silently duplicates frames, are covered in generating video thumbnails with FFmpeg in Node.js.

EXIF orientation and stripped colour profiles #

Sharp does not auto-rotate. A portrait photo from any iPhone is stored landscape with an orientation tag, so without .rotate() your thumbnails are sideways — and only for some users, which makes the bug report confusing. Sharp also strips all metadata by default, including the ICC profile; a Display-P3 photo re-encoded without one renders desaturated in every browser. .rotate().withMetadata({ icc: "srgb" }) fixes both, and deliberately drops GPS EXIF, which you should not be republishing anyway.

/tmp exhaustion on Lambda #

The default 512 MB holds a source file and nothing else. Downloading a 400 MB MOV and writing two renditions needs about 1.5 GB, and the failure mode is Error: ENOSPC: no space left on device, write. Worse, /tmp persists across warm invocations, so a leaked temp file from a crashed job shrinks the budget for the next one until the sandbox recycles. Set EphemeralStorage to source-size plus twice the expected output, and delete the work directory in a finally block.

Retry storms when a backlog drains #

A queue that accumulated 200,000 messages during an outage will, the moment the workers come back, try to run all of them at once. Lambda scales to your account concurrency and every one of those invocations issues a GetObject against the same date prefix. You will see SlowDown (HTTP 503) from S3, connection-pool exhaustion against Postgres, and a retry cascade that looks like a second outage. Cap the event source’s MaximumConcurrency — 10 is a sane starting point for video — and let the backlog drain in half an hour rather than melting the database.

Recipes that change without a version bump #

Editing a quality setting and redeploying produces derivatives whose bytes differ from the ones already cached with max-age=31536000, immutable. Users see a mix indefinitely. Every recipe change must bump version, which changes the job key and the output prefix. Making version part of the key is what turns “we changed the thumbnail size” from an incident into a backfill job.

Verification #

Confirm the trigger is wired, then prove a real object round-trips.

# 1. The bucket must be emitting to EventBridge.
aws s3api get-bucket-notification-configuration --bucket uploads-prod
# Expect: { "EventBridgeConfiguration": {} }

# 2. Drop a real source in and watch the queue pick it up.
aws s3 cp ./probe.mov s3://uploads-prod/originals/2026/07/26/probe/source.mov
sleep 5
aws sqs get-queue-attributes --queue-url "$QUEUE_URL" \
  --attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible
# Expect at least one message visible or in flight within ~5 s.

# 3. The derivative set exists under the versioned prefix.
aws s3 ls s3://uploads-prod/derivatives/probe/v3/ --recursive
# Expect: 720p.mp4, 360p.mp4, poster_1280.jpg, manifest.json

# 4. Nothing landed in the dead-letter queue.
aws sqs get-queue-attributes --queue-url "$DLQ_URL" \
  --attribute-names ApproximateNumberOfMessages
# Expect: "0"

Then assert on the output itself rather than on its existence, because a 12-byte MP4 also “exists”:

ffprobe -v error -show_entries format=duration,format_name \
  -show_entries stream=codec_name,width,height -of json 720p.mp4

{ "streams": [ { "codec_name": "h264", "width": 1280, "height": 720 },
               { "codec_name": "aac" } ],
  "format": { "duration": "62.041000", "format_name": "mov,mp4,m4a,3gp,3g2,mj2" } }

Finally, prove the idempotency guard actually guards. Replay the same event body twice and assert the second run does no work:

import assert from "node:assert/strict";

const src = { bucket: "uploads-prod", key: "originals/2026/07/26/probe/source.mov", versionId: "3HL4kqCxf3vjVBH40Nrjfkd" };
const key = jobKey(src, { id: "web-video", version: 3 });

assert.equal(key, jobKey(src, { id: "web-video", version: 3 }), "job key must be deterministic");
assert.notEqual(key, jobKey(src, { id: "web-video", version: 4 }), "a recipe bump must produce a new key");

assert.equal(await claimJob(key, src, VIDEO_RECIPE), "claimed");
assert.equal(await claimJob(key, src, VIDEO_RECIPE), "duplicate");
console.log("idempotency guard verified");

If the second claimJob returns "claimed", your ON CONFLICT predicate is wrong and you are paying twice for every duplicated event — which, at S3’s delivery guarantees, is a measurable fraction of your bill.

Frequently Asked Questions