Generating Animated Previews from Video #

Make hover previews by sampling short segments across the timeline — for example six 1-second clips at 10 %, 25 %, 40 %, 55 %, 70 % and 85 % of the duration — and concatenating them with FFmpeg’s select and setpts filters into a muted, 320–480 px wide H.264 MP4 of about six seconds (typically 100–300 KB), with an animated WebP fallback for image-only contexts. For scrubbing thumbnails, extract one frame every few seconds into a tiled sprite sheet with fps and tile, and write a WebVTT file mapping each time range to its tile’s #xywh= coordinates so players show the right frame on hover.

Animated previews make video libraries browsable: users see what a clip contains without starting it, and sprite-sheet scrubbing makes seeking precise. Both are derivatives you generate once after upload. Choosing the format matters more than it seems: GIF previews are ten times larger than equivalent MP4s and look worse. This page belongs to post-upload media transcoding in backend validation and cloud storage architecture; single poster frames are covered in generating video thumbnails with FFmpeg in Node.js.

When to use this approach #

• Grids of videos where hover or autoplay-in-view previews help users choose.
• Players that show a thumbnail while the user drags the seek bar.
• Anywhere you are tempted to generate GIFs — the MP4 loop is smaller and sharper.

Prerequisites #

1. FFmpeg 6+ with libx264 and libwebp (ffmpeg -encoders | grep -E 'libx264|libwebp').
2. The source duration and dimensions from the probe step (validating video uploads with ffprobe).
3. A processing worker — Lambda is a good fit, as in running FFmpeg in AWS Lambda with container images.

Two derivatives, two purposes #

Implementation #

import { spawn } from "node:child_process";
import { writeFile } from "node:fs/promises";

function run(args: string[], timeoutMs = 120_000): Promise<void> {
  return new Promise((resolve, reject) => {
    const p = spawn("ffmpeg", ["-hide_banner", "-nostdin", "-y", "-v", "error", ...args]);
    let err = ""; p.stderr.on("data", (d) => (err += d));
    const t = setTimeout(() => p.kill("SIGKILL"), timeoutMs);
    p.on("close", (c) => { clearTimeout(t); c === 0 ? resolve() : reject(new Error(err.slice(-2000))); });
  });
}

/** Six 1 s segments spread across the video, joined into one muted loop. */
export async function hoverPreview(src: string, duration: number, out: { mp4: string; webp: string }) {
  const short = duration < 8;                       // short clips: just take the first six seconds
  const points = [0.1, 0.25, 0.4, 0.55, 0.7, 0.85].map((f) => f * duration);
  const select = points.map((t) => `between(t,${t.toFixed(2)},${(t + 1).toFixed(2)})`).join("+");
  const scale = "fps=24,scale=480:-2:flags=lanczos";
  const vf = short ? scale : `select='${select}',setpts=N/FRAME_RATE/TB,${scale}`;

  const inputArgs = short ? ["-t", "6", "-i", src] : ["-i", src];
  await run([...inputArgs, "-an", "-vf", vf,
    "-c:v", "libx264", "-preset", "slow", "-crf", "28", "-profile:v", "main", "-pix_fmt", "yuv420p",
    "-movflags", "+faststart", out.mp4]);

  // Animated WebP from the finished MP4 (fast: it is already short and small).
  await run(["-i", out.mp4, "-vf", "fps=12,scale=320:-2:flags=lanczos",
    "-c:v", "libwebp", "-lossless", "0", "-q:v", "60", "-loop", "0", "-an", out.webp]);
}

/** Sprite sheet of frames every `interval` seconds, plus a WebVTT cue file. */
export async function scrubSprite(src: string, duration: number, spriteUrl: string, out: { jpg: string; vtt: string }) {
  const interval = duration <= 120 ? 2 : duration <= 1200 ? 5 : 10;
  const count = Math.ceil(duration / interval);
  const cols = 10, rows = Math.ceil(count / cols), tw = 160, th = 90;

  await run(["-i", src, "-an",
    "-vf", `fps=1/${interval},scale=${tw}:${th}:force_original_aspect_ratio=decrease,pad=${tw}:${th}:(ow-iw)/2:(oh-ih)/2,tile=${cols}x${rows}`,
    "-frames:v", "1", "-q:v", "5", out.jpg], 300_000);

  const ts = (s: number) => new Date(s * 1000).toISOString().slice(11, 23);
  let vtt = "WEBVTT\n\n";
  for (let i = 0; i < count; i++) {
    const x = (i % cols) * tw, y = Math.floor(i / cols) * th;
    vtt += `${ts(i * interval)} --> ${ts(Math.min((i + 1) * interval, duration))}\n${spriteUrl}#xywh=${x},${y},${tw},${th}\n\n`;
  }
  await writeFile(out.vtt, vtt);
}

Line-by-line on the decisions that matter #

• select with between() segments and setpts=N/FRAME_RATE/TB. select keeps only frames inside the chosen windows; setpts renumbers them so the output plays continuously instead of jumping in time with gaps. One decode pass produces the whole loop.
• Short videos handled separately. For clips under eight seconds, sampling six windows produces overlapping, jittery output. Taking the first six seconds is better.
• MP4 at CRF 28, -preset slow. Previews are tiny, so a slow preset costs seconds and saves bytes on every page view. yuv420p and main profile keep it playable in every browser.
• Muted. -an removes audio. Browsers only autoplay muted video, and hover previews should never make sound.
• WebP from the MP4. Re-encoding the already short, already scaled loop is much cheaper than working from the source. Animated WebP is supported in all current browsers and is the right fallback for contexts that only accept images — email-like previews, <img>-only CMSs.
• tile filter for sprites. The frames are padded to a fixed tile size so the VTT coordinates are simple multiples. One JPEG request serves every scrub thumbnail.
• Interval by duration. Fixed 2-second sampling makes a two-hour video’s sprite enormous. Scaling the interval keeps sprites under roughly 100 tiles per sheet for most content; very long videos can use several sheets.

Why not GIF #

GIF has no inter-frame compression worth the name and a 256-colour palette. A muted, looping <video autoplay muted loop playsinline> behaves like a GIF from the user’s point of view and is an order of magnitude smaller. If a destination truly only accepts GIF (some chat integrations), generate it on demand from the MP4 with a palette pass (palettegen/paletteuse) rather than storing one for every upload.

Using the derivatives in the page #

Load hover previews lazily. Render the poster image in the grid; on pointer enter (or when the card scrolls into view on touch devices), set the <video> element’s src to the preview and call play(). Unload it on pointer leave to free memory in long grids. Respect prefers-reduced-motion by not autoplaying at all for users who ask for less motion — show the static poster and play only on explicit interaction.

For scrubbing, most players read the VTT directly: Video.js with a thumbnails plugin, Shaka Player’s image tracks, Vidstack and Plyr all accept a WebVTT thumbnail track with #xywh fragments. Serve the sprite and VTT from the same CDN path as the video renditions, with long cache lifetimes, because they never change once generated.

Choosing sample points well #

Evenly spaced windows are a good default but can land on black frames, fades or title cards. Two cheap improvements help. First, skip the first and last few percent of the timeline, where intros and credits live — the fractions above already start at 10 % and end at 85 %. Second, nudge each window to a nearby “interesting” frame by running FFmpeg’s thumbnail filter over a few seconds around each point, or by using scene-change scores (select='gt(scene,0.3)') to prefer windows that contain motion. For user-generated content, the simple spacing is usually good enough; for a catalogue where previews drive clicks, the extra analysis pass is worth it.

Storing and naming the derivatives #

Keep preview artefacts next to the other renditions of the same video, keyed by the source’s content hash or job ID rather than the user’s filename: derived/<id>/preview.mp4, preview.webp, sprite.jpg and sprite.vtt. Record their existence and dimensions in the metadata table so the front end knows whether a preview is available before requesting it, and can fall back to the poster for videos processed before previews existed. When you change preview settings — a new width, a different number of segments — version the key (preview-v2.mp4) and regenerate in the background, so cached pages keep working while the new files roll out.

Configuration gotchas #

The preview stutters or has frozen frames. Variable frame rate sources produce uneven output after select. The fps=24 filter after setpts normalises the frame rate.

Sprite tiles are misaligned with the VTT. Rounding in scale changes tile size for some aspect ratios. The force_original_aspect_ratio=decrease,pad pair guarantees exact tile dimensions.

tile produces a sheet with empty tiles at the end. Rows are rounded up; the last row may be partially empty. That is harmless — the VTT only references tiles that exist.

Previews don’t autoplay on iOS. Add playsinline and muted attributes to the video element; iOS requires both for inline autoplay.

Verification #

ffprobe -v error -show_entries format=duration,size:stream=width,height,codec_name -of compact preview.mp4
# stream|codec_name=h264|width=480|height=270  format|duration=6.000000|size=201344
head -8 sprite.vtt
# WEBVTT
# 00:00:00.000 --> 00:00:05.000
# https://cdn.example.com/derived/abc/sprite.jpg#xywh=0,0,160,90

Frequently Asked Questions