Generating Waveform Peaks for Audio Players #

Decode the upload to mono 16-bit PCM with FFmpeg, stream it through a reducer that keeps the minimum and maximum sample in each fixed-size bucket (for example 800 buckets for the whole file, or one bucket per 256 samples for zoomable views), store the resulting pairs as a small JSON or Int8Array file beside the audio, and let the player draw it immediately instead of decoding megabytes of audio in the browser.

Decoding audio client-side just to draw a waveform is one of the most expensive things an audio page can do: AudioContext.decodeAudioData on a 60-minute podcast allocates about 1.2 GB of Float32 samples and freezes low-end phones. Peaks computed once at processing time weigh a few kilobytes and render in a millisecond. This page belongs to audio processing pipelines in media processing and delivery pipelines. Compute peaks after normalizing audio loudness with FFmpeg loudnorm so the drawn shape matches what listeners hear.

When to use this approach #

• Your player shows a waveform — a podcast app, a voice-note feed, an audio editor’s overview — and must render before or without playback.
• Files are long enough that browser-side decoding is slow or memory-hungry: anything over a couple of minutes.
• You process uploads server-side already, so one more pass over decoded PCM is cheap.

Prerequisites #

1. FFmpeg 5+ on the worker, used only to decode to raw PCM on stdout.
2. Node 20+; the reducer uses streams and typed arrays, no native modules.
3. A decision on resolution: a fixed bucket count for overview waveforms (800–2000), or samples-per-pixel levels for zoom.
4. Storage next to the audio — audio/<assetId>/peaks.json — served with the same caching as the audio file.

Why min and max, not an average #

A waveform bar represents thousands of samples. Averaging them gives nearly zero, because audio oscillates around zero. RMS gives loudness but hides transients. Storing the minimum and maximum of each bucket preserves the envelope a listener recognises — plosives, laughs, the drum hit at the chorus — and drawing a vertical line from min to max per pixel reproduces exactly what an audio editor shows.

Implementation #

The worker streams decoded PCM straight from FFmpeg’s stdout, so memory stays flat regardless of file length.

import { spawn } from "node:child_process";
import { writeFile } from "node:fs/promises";

export interface Peaks {
  version: 2;
  sampleRate: number;
  samplesPerPixel: number;   // bucket size in samples
  bits: 8;                   // values scaled to -128..127
  length: number;            // number of (min, max) pairs
  durationS: number;
  data: number[];            // interleaved [min0, max0, min1, max1, …]
}

const SAMPLE_RATE = 22_050;  // half of 44.1k is plenty for a visual envelope

/** Decode to mono s16le PCM and reduce to min/max pairs per bucket. */
export async function computePeaks(input: string, samplesPerPixel = 256): Promise<Peaks> {
  const ff = spawn("ffmpeg", [
    "-hide_banner", "-nostats", "-i", input,
    "-vn", "-ac", "1", "-ar", String(SAMPLE_RATE),
    "-f", "s16le", "-acodec", "pcm_s16le", "pipe:1",
  ], { stdio: ["ignore", "pipe", "pipe"] });

  let stderr = "";
  ff.stderr.on("data", (d: Buffer) => { stderr = (stderr + d.toString()).slice(-2000); });

  const out: number[] = [];
  let min = 32767, max = -32768, count = 0, total = 0;
  let carry: Buffer | null = null;          // an odd byte left over between chunks

  for await (const chunk of ff.stdout as AsyncIterable<Buffer>) {
    const buf: Buffer = carry ? Buffer.concat([carry, chunk]) : chunk;
    const usable = buf.length - (buf.length % 2);
    carry = usable < buf.length ? buf.subarray(usable) : null;

    for (let i = 0; i < usable; i += 2) {
      const s = buf.readInt16LE(i);
      if (s < min) min = s;
      if (s > max) max = s;
      if (++count === samplesPerPixel) {
        out.push(min >> 8, max >> 8);         // 16-bit → 8-bit, keeps the sign
        min = 32767; max = -32768; count = 0;
      }
    }
    total += usable / 2;
  }
  if (count > 0) out.push(min >> 8, max >> 8);  // final partial bucket

  const code: number = await new Promise((r) => ff.on("close", r));
  if (code !== 0) throw new Error(`ffmpeg exited ${code}: ${stderr}`);

  return {
    version: 2,
    sampleRate: SAMPLE_RATE,
    samplesPerPixel,
    bits: 8,
    length: out.length / 2,
    durationS: total / SAMPLE_RATE,
    data: out,
  };
}

/** Derive a fixed-width overview (e.g. 800 bars) from the fine peaks, no re-decode. */
export function resample(p: Peaks, bars: number): number[] {
  const out: number[] = [];
  const per = p.length / bars;
  for (let b = 0; b < bars; b++) {
    const from = Math.floor(b * per), to = Math.max(from + 1, Math.floor((b + 1) * per));
    let lo = 127, hi = -128;
    for (let i = from; i < to && i < p.length; i++) {
      lo = Math.min(lo, p.data[2 * i]);
      hi = Math.max(hi, p.data[2 * i + 1]);
    }
    out.push(lo, hi);
  }
  return out;
}

// Usage: node --experimental-strip-types peaks.ts normalised.wav peaks.json
if (process.argv[2] && process.argv[3]) {
  const peaks = await computePeaks(process.argv[2]);
  await writeFile(process.argv[3], JSON.stringify({ ...peaks, overview: resample(peaks, 800) }));
  console.log(`${peaks.length} pairs, ${peaks.durationS.toFixed(1)} s`);
}

Line-by-line on the parameters that matter #

• -ac 1 -ar 22050. Downmixing to mono and halving the rate cuts the data FFmpeg writes and Node reads by 4×, and a visual envelope does not need frequencies above 11 kHz. Keep stereo only if you draw channels separately.
• s16le on a pipe. Raw little-endian 16-bit PCM is the cheapest format to parse — two bytes per sample, no headers. Reading it with readInt16LE in a loop is fast enough for real-time-multiple decoding in plain JavaScript.
• The carry byte. Stream chunks can end in the middle of a two-byte sample. Dropping or misaligning that byte shifts every subsequent sample by one byte and produces noise. Carrying the odd byte into the next chunk keeps alignment.
• >> 8 to 8-bit. A display at most a few hundred pixels tall cannot show 65,536 levels. Eight bits (−128 to 127) is the format BBC’s audiowaveform and peaks.js use, halves the file, and keeps sign via arithmetic shift.
• samplesPerPixel = 256. At 22.05 kHz that is 86 pairs per second: detailed enough to zoom into individual words. The resample helper derives any coarser overview from it without decoding again.
• Peaks from the normalised file. A waveform drawn from the original upload of a quiet memo would be a flat line; drawn after normalisation it matches the level the listener hears.

Drawing it in the browser #

export function drawWaveform(canvas: HTMLCanvasElement, overview: number[], progress = 0): void {
  const dpr = window.devicePixelRatio || 1;
  const w = canvas.clientWidth, h = canvas.clientHeight;
  canvas.width = Math.round(w * dpr);
  canvas.height = Math.round(h * dpr);
  const ctx = canvas.getContext("2d");
  if (!ctx) return;
  ctx.scale(dpr, dpr);
  const bars = overview.length / 2, bw = w / bars, mid = h / 2;
  for (let i = 0; i < bars; i++) {
    const lo = overview[2 * i] / 128, hi = overview[2 * i + 1] / 128;
    ctx.fillStyle = i / bars < progress ? "#7a1515" : "#c97c1a";
    ctx.fillRect(i * bw, mid - hi * mid, Math.max(1, bw - 1), Math.max(1, (hi - lo) * mid));
  }
}

Configuration gotchas #

Waveform looks like noise, symmetrical and full-height. Byte misalignment: a chunk boundary split a sample and the reader never recovered. Check the carry logic, and make sure you are reading readInt16LE, not big-endian — FFmpeg’s s16le is little-endian on every platform.

pipe:1: Invalid argument on Windows workers. FFmpeg needs -f s16le explicitly when writing to a pipe; it cannot infer a format from a pipe name. The command above sets it.

Peaks and audio drift apart when seeking. The peaks were computed from a file with a different duration than the one being played — typically the original instead of the transcoded AAC, which gains encoder priming samples. Compute peaks from the same file you serve, or store durationS and scale by the player’s duration.

The waveform is flat for quiet uploads. You computed peaks before normalising. Either compute them after, or scale the drawing by the maximum peak in the file.

Size of peak files by resolution #

Verification #

import { strict as assert } from "node:assert";
import { computePeaks, resample } from "./peaks.ts";

// A 1 kHz sine at -6 dBFS, 2 s — generate with:
// ffmpeg -f lavfi -i "sine=frequency=1000:duration=2" -af "volume=-6dB" sine.wav
const p = await computePeaks("fixtures/sine.wav");
assert.ok(Math.abs(p.durationS - 2) < 0.01, "duration preserved");
const maxes = p.data.filter((_, i) => i % 2 === 1);
assert.ok(maxes.every((m) => m >= 60 && m <= 66), "-6 dBFS ≈ 0.5 × 127 ≈ 63");
assert.equal(resample(p, 100).length, 200, "100 bars → 200 values");
console.log("peaks ok:", p.length, "pairs");

A generated sine wave is the ideal fixture: its peaks are known exactly, so any alignment or scaling bug shows up as a number, not as a subjective “looks wrong”.

Frequently Asked Questions