Generating srcset Variants at Upload Time #

Derive a fixed width ladder from the widths your layouts actually render at (times 1× and 2× device pixel ratio), generate every width that is not larger than the original with Sharp in one pass over a single decoded buffer, record which widths exist, and build srcset and sizes from that record — never from assumptions.

srcset only saves bytes if the candidate widths are close to what the browser needs. A ladder of 400w, 800w, 1600w for an image rendered 360 CSS pixels wide on a 3× phone makes the browser pick the 1600 px file for a 1080-device-pixel slot — a 48% overshoot on every feed image. This page is part of responsive image delivery in media processing and delivery pipelines. The derivative worker it extends is described in building an image derivative pipeline with Sharp.

When to use this approach #

• Images are user uploads displayed in a small number of known layouts — feed card, detail page, avatar, lightbox.
• You would rather pay compute once at upload than per request at the edge (compare resizing images on the fly with a Cloudflare Worker).
• You want variants stored beside the original so a CDN can serve them with immutable caching.

Prerequisites #

1. Node 20+ and sharp 0.33+ (npm i sharp), installed on the same OS and CPU architecture you deploy to.
2. An object store with the original under uploads/<assetId>/original and write access to img/<assetId>/.
3. A list of your layouts’ rendered widths in CSS pixels, from the design system or measured in DevTools.
4. A place to persist the generated widths — a JSON column on the asset row is enough.

Choosing the width ladder #

Start from the slots, not from round numbers. If your layouts render images at 64 (avatar), 320 (feed card on mobile), 400 (card on desktop grid), 720 (detail column) and 1200 (lightbox) CSS pixels, the candidates you need are each of those at 1× and 2×, deduplicated and rounded up to a small set. Three-times density is common on phones, but a 3× candidate rarely beats a 2× one visibly for photographs, and serving 2× to 3× screens saves roughly 45% of bytes.

The largest candidate should rarely exceed 2000 px: phone photos are 4000 px wide, but no layout on a normal page shows one at that density, and a 2000 px AVIF is already a few hundred kilobytes.

Implementation #

import sharp from "sharp";
import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({});
const BUCKET = process.env.MEDIA_BUCKET!;

export const LADDER = [160, 320, 480, 640, 800, 1200, 1600, 2000] as const;

const FORMATS = {
  avif: { ext: "avif", type: "image/avif", opts: { quality: 50, effort: 4 } },
  webp: { ext: "webp", type: "image/webp", opts: { quality: 75, effort: 4 } },
  jpeg: { ext: "jpg", type: "image/jpeg", opts: { quality: 80, mozjpeg: true } },
} as const;

export interface VariantRecord {
  widths: number[];            // widths that exist for every format
  aspect: number;              // height / width of the oriented original
  formats: (keyof typeof FORMATS)[];
}

async function readOriginal(key: string): Promise<Buffer> {
  const res = await s3.send(new GetObjectCommand({ Bucket: BUCKET, Key: key }));
  return Buffer.from(await res.Body!.transformToByteArray());
}

export async function generateVariants(assetId: string): Promise<VariantRecord> {
  const input = await readOriginal(`uploads/${assetId}/original`);

  // Decode once, apply EXIF orientation, strip metadata. limitInputPixels guards pixel bombs.
  const base = sharp(input, { limitInputPixels: 100_000_000 }).rotate();
  const meta = await base.metadata();
  const oriented = (meta.orientation ?? 1) >= 5;          // 5–8 swap width and height
  const srcW = (oriented ? meta.height : meta.width) ?? 0;
  const srcH = (oriented ? meta.width : meta.height) ?? 0;
  if (!srcW || !srcH) throw new Error("unreadable image dimensions");

  // Never upscale. Always keep at least the smallest rung so tiny uploads still get a variant.
  const widths = LADDER.filter((w) => w <= srcW);
  if (widths.length === 0) widths.push(Math.min(srcW, LADDER[0]));

  const jobs: Promise<unknown>[] = [];
  for (const w of widths) {
    // One resize per width, cloned per format — decode and resize happen once per width.
    const resized = base.clone().resize({ width: w, withoutEnlargement: true });
    for (const [name, f] of Object.entries(FORMATS)) {
      jobs.push(
        resized.clone()
          .toFormat(name as keyof typeof FORMATS, f.opts)
          .toBuffer()
          .then((body) => s3.send(new PutObjectCommand({
            Bucket: BUCKET,
            Key: `img/${assetId}/${w}.${f.ext}`,
            Body: body,
            ContentType: f.type,
            CacheControl: "public, max-age=31536000, immutable",
          }))),
      );
    }
  }
  await Promise.all(jobs);

  return { widths, aspect: srcH / srcW, formats: ["avif", "webp", "jpeg"] };
}

/** Build srcset/sizes from what actually exists — never from the full ladder. */
export function imgAttributes(
  assetId: string,
  rec: VariantRecord,
  sizes: string,
  base = "https://img.example.com",
): { src: string; srcset: string; sizes: string; width: number; height: number } {
  const srcset = rec.widths.map((w) => `${base}/img/${assetId}/${w} ${w}w`).join(", ");
  const fallback = rec.widths.find((w) => w >= 640) ?? rec.widths[rec.widths.length - 1];
  return {
    src: `${base}/img/${assetId}/${fallback}`,
    srcset,
    sizes,
    width: fallback,
    height: Math.round(fallback * rec.aspect),   // reserves layout space: no CLS
  };
}

// Usage
const rec = await generateVariants("9c1f2a7e-44b0");
console.log(imgAttributes("9c1f2a7e-44b0", rec, "(min-width: 1024px) 400px, calc(100vw - 32px)"));

Line-by-line on the parameters that matter #

• .rotate() with no argument applies EXIF orientation and drops the tag. Skip it and portrait phone photos come out sideways in every variant, because resizing preserves pixels, not the flag. It also strips the metadata — see stripping EXIF metadata before upload for why GPS tags should never reach a public variant.
• limitInputPixels: 100_000_000 makes Sharp refuse images over 100 megapixels before allocating memory for them — the defence discussed in validating image dimensions and pixel bombs server-side.
• widths = LADDER.filter((w) => w <= srcW) — never generate a width larger than the original. A 900 px upload gets 160 to 800; there is no 1200 file pretending to be sharper than its source.
• clone() per width, then per format. Sharp pipelines are lazy; cloning shares the decoded input so the original is decoded once rather than 24 times. For a 12-megapixel JPEG that is the difference between about 1.5 and 9 seconds of CPU.
• Quality values differ per format because the scales are not comparable: AVIF 50 roughly matches WebP 75 and JPEG 80 on photographs. Choosing quality settings for AVIF, WebP and JPEG shows how to calibrate these against your own uploads.
• width and height attributes from aspect. Browsers compute the aspect ratio from these before the image loads, so the layout never jumps — the biggest cheap win for Cumulative Layout Shift on image-heavy feeds.

How sizes decides the download #

srcset lists what exists; sizes tells the browser how wide the slot will be before layout, so it can pick a candidate from the preload scanner. Get sizes wrong and a correct ladder is wasted.

Configuration gotchas #

Error: Input image exceeds pixel limit. Sharp’s limitInputPixels fired. That is the guard working; reject the upload with a clear message rather than raising the limit globally. If you genuinely accept panoramas, raise it for that upload type only.

VipsJpeg: Corrupt JPEG data: premature end of data segment as a warning, followed by grey bands in variants. The upload was truncated — often an interrupted multipart assembly. Sharp decodes what it can by default; pass failOn: "truncated" in the constructor options to fail fast instead of publishing a half-grey image.

Could not load the "sharp" module using the linux-x64 runtime. You installed on macOS or ARM and deployed to x64 Lambda. Install with npm install --os=linux --cpu=x64 sharp in the build step, or build inside the target container image.

Variants look washed out. The original had a Display P3 or Adobe RGB ICC profile and .rotate() stripped it without converting. Add .toColourspace("srgb") (or .withMetadata({ icc: "srgb" })) before encoding so colours are converted rather than reinterpreted.

The cost of the ladder per upload #

Verification #

# Every width up to the original exists in every format — and nothing larger.
aws s3 ls "s3://$MEDIA_BUCKET/img/9c1f2a7e-44b0/" | awk '{print $4}' | sort -V

# Variants are upright and metadata-free.
aws s3 cp "s3://$MEDIA_BUCKET/img/9c1f2a7e-44b0/800.jpg" - | exiftool -Orientation -GPSLatitude -
# (no output = no orientation tag and no GPS)

In the browser, load a page using the attributes, open DevTools → Network, filter by “Img”, and check the Size column against the slot: a 400 CSS px card on a 2× laptop should fetch the 800 candidate. Then resize the window and confirm a larger candidate is only fetched when the slot grows, never on shrink.

Frequently Asked Questions