Generating Video Thumbnails in the Browser #

Load the selected file into a detached <video> element via URL.createObjectURL(file) with muted, playsInline and preload="metadata", wait for loadedmetadata, seek to a point a little into the video (10 % of the duration, capped at a few seconds), wait for seeked — and on browsers that support it, requestVideoFrameCallback so the frame is actually decoded — then draw the video onto a canvas sized to the thumbnail and export with canvas.toBlob("image/jpeg", 0.8). Reject frames that are almost entirely black by sampling pixel brightness and try a later point, time out after a few seconds, and fall back to a generic video icon when the browser cannot decode the codec. Revoke the object URL as soon as the frame is captured.

Users choosing a video for upload want to see that they picked the right one. Waiting for the upload and server-side processing to finish before showing a thumbnail can take minutes for large files. The browser can already decode most phone videos, so a frame is available within a second of selection — enough for the upload list, a cover-image picker, or a local preview while the upload runs. This page belongs to client-side media preprocessing in frontend UX, chunking and progress tracking; the authoritative server-side thumbnails are covered in generating video thumbnails with FFmpeg in Node.js.

When to use this approach #

• Users upload videos and see them in a list or grid before processing finishes.
• You want to let users pick a cover frame before uploading.
• You need a quick sanity check that a file is a playable video.

Prerequisites #

1. Evergreen browsers; requestVideoFrameCallback is available in Chromium and Safari and improves reliability where present.
2. Knowledge of which codecs your users upload: H.264 and HEVC from phones (HEVC decodes in Safari and in Chromium on hardware that supports it), VP9 and AV1 from screen recorders.
3. Object URL handling from previewing files with object URLs.

How the capture works #

Implementation #

interface Thumb { blob: Blob; width: number; height: number; duration: number; atSeconds: number }

function once<T extends Event>(el: EventTarget, ok: string, ms: number): Promise<T> {
  return new Promise((resolve, reject) => {
    const t = setTimeout(() => { cleanup(); reject(new Error(`timeout waiting for ${ok}`)); }, ms);
    const onOk = (e: Event) => { cleanup(); resolve(e as T); };
    const onErr = () => { cleanup(); reject(new Error("media error")); };
    const cleanup = () => { clearTimeout(t); el.removeEventListener(ok, onOk); el.removeEventListener("error", onErr); };
    el.addEventListener(ok, onOk, { once: true });
    el.addEventListener("error", onErr, { once: true });
  });
}

function nextFrame(video: HTMLVideoElement): Promise<void> {
  // Ensures the seeked frame is decoded and presented before drawing.
  if ("requestVideoFrameCallback" in video) {
    return new Promise((r) => (video as any).requestVideoFrameCallback(() => r()));
  }
  return new Promise((r) => requestAnimationFrame(() => requestAnimationFrame(() => r())));
}

function isMostlyDark(ctx: CanvasRenderingContext2D, w: number, h: number): boolean {
  const { data } = ctx.getImageData(0, 0, w, h);
  let bright = 0, samples = 0;
  for (let i = 0; i < data.length; i += 4 * 97) {        // sample roughly every 97th pixel
    const luma = 0.2126 * data[i] + 0.7152 * data[i + 1] + 0.0722 * data[i + 2];
    if (luma > 40) bright++;
    samples++;
  }
  return bright / samples < 0.05;
}

export async function videoThumbnail(file: File, maxWidth = 480): Promise<Thumb | null> {
  const url = URL.createObjectURL(file);
  const video = document.createElement("video");
  video.muted = true; video.playsInline = true; video.preload = "metadata"; video.src = url;
  try {
    await once(video, "loadedmetadata", 5000);
    const duration = Number.isFinite(video.duration) ? video.duration : 0;
    const scale = Math.min(1, maxWidth / video.videoWidth);
    const w = Math.round(video.videoWidth * scale), h = Math.round(video.videoHeight * scale);
    if (!w || !h) return null;                               // audio-only or undecodable video track

    const canvas = document.createElement("canvas");
    canvas.width = w; canvas.height = h;
    const ctx = canvas.getContext("2d", { willReadFrequently: true })!;

    const candidates = [Math.min(3, duration * 0.1), duration * 0.25, duration * 0.5].filter((t, i, a) => t >= 0 && a.indexOf(t) === i);
    for (const at of candidates) {
      video.currentTime = at;
      await once(video, "seeked", 4000);
      await nextFrame(video);
      ctx.drawImage(video, 0, 0, w, h);
      if (!isMostlyDark(ctx, w, h) || at === candidates.at(-1)) {
        const blob = await new Promise<Blob | null>((r) => canvas.toBlob(r, "image/jpeg", 0.8));
        return blob ? { blob, width: w, height: h, duration, atSeconds: at } : null;
      }
    }
    return null;
  } catch {
    return null;                                             // unsupported codec or corrupt file: caller shows a placeholder
  } finally {
    video.removeAttribute("src"); video.load();              // release the decoder
    URL.revokeObjectURL(url);
  }
}

Line-by-line on the decisions that matter #

• muted and playsInline. Mobile browsers restrict media elements that could play sound or go full screen. A muted, inline element can load and seek without user interaction in every current browser.
• preload="metadata" then seek. Only metadata and the frames around the seek point are decoded, not the whole file. For a 2 GB video, the thumbnail still arrives in about a second.
• Seek to 10 %, capped at three seconds. Frame zero is often black, a fade-in or a camera focusing. A point a few seconds in is usually representative, and the cap keeps seeks short for long videos.
• requestVideoFrameCallback after seeked. Some browsers fire seeked before the new frame is ready to paint, and drawing then captures the previous (or a blank) frame. Waiting for the next presented frame fixes that; two animation frames are the fallback.
• Dark-frame check. Sampling brightness on a subset of pixels is cheap and catches the common black-frame cases, falling back to 25 % and 50 % of the duration.
• Timeouts on every wait. When the browser cannot decode a codec (HEVC on some platforms, ProRes almost everywhere), events never fire. A timeout returns null quickly so the UI can show a placeholder.
• Release everything. Clearing src and calling load() releases the decoder immediately, and revoking the object URL frees the reference. Without this, a batch of fifty videos can exhaust hardware decoders on mobile devices.

Codec support in practice #

Letting users choose a cover frame #

The same technique supports a cover picker: capture five or six frames at evenly spaced times into a strip, let the user choose one, and upload the chosen frame as a separate small image alongside the video (or send its timestamp and let the server extract the frame at full quality). Capturing the frames sequentially, not in parallel, keeps memory steady: one video element, one canvas, seek-draw-export in a loop. Label the strip with timestamps for keyboard and screen-reader users, and make each frame a real button with an accessible name like “Frame at 0:42”.

Sending the timestamp is usually better than uploading the browser’s frame. The server extracts from the original at full resolution with consistent colour handling, while the browser’s frame depends on its decoder and may be scaled. Use the browser frame for immediate display and the timestamp for the durable result.

Configuration gotchas #

The thumbnail is black on iOS. The frame was drawn before it was decoded. Wait for requestVideoFrameCallback (available since iOS 15.4) after seeked, and make sure the element is muted and playsInline.

Thumbnails are rotated for portrait phone videos. Browsers apply the rotation metadata when rendering a video element and when drawing it to canvas in current versions. If you see sideways frames, the browser is old; show the placeholder instead of a wrong image.

SecurityError: The operation is insecure when exporting. The video came from a cross-origin URL without CORS. Local files via object URLs never taint the canvas; this only appears if you reuse the code for remote videos.

Batch of many videos freezes the page. Each capture holds a decoder. Generate thumbnails one or two at a time, and only for rows that are visible.

Verification #

• Choose an iPhone HEVC video in Safari: a thumbnail appears in under a second and is not black.
• Choose the same file in Firefox on a system without HEVC support: the placeholder appears within the timeout.
• Choose a video that starts with two seconds of black: the thumbnail comes from 25 % of the duration.
• Take a memory snapshot after 30 thumbnails: no detached video elements or unrevoked blob URLs remain.

Frequently Asked Questions