Real-Time Upload Progress Events #

A progress bar is a promise to the user, and a bar that freezes at 99% or leaps from 30% to done breaks that promise even when the transfer succeeded. Honest progress means separating two signals — bytes leaving the browser, which the client can measure, and server-side work like scanning and transcoding, which only the backend can report — then combining them into one throttled, monotonic display.

This topic lives under Frontend UX, Chunking & Progress Tracking and reads the same committed offset produced by resumable upload state machines. Everything below assumes a chunked, direct-to-storage upload, but the aggregation and smoothing logic works unchanged for a single PUT.

Prerequisites #

• [ ] Node 20+ and a bundler that emits ESM, for building the client module
• [ ] TypeScript 5.x with strict enabled — the aggregator relies on non-nullable maps
• [ ] A file already split into parts, as covered in slicing large files with Blob.slice
• [ ] An endpoint that streams processing events as text/event-stream, or a WebSocket equivalent
• [ ] Access-Control-Expose-Headers: ETag on the storage bucket, or getResponseHeader("ETag") returns null
• [ ] A UI layer where you can batch state writes into requestAnimationFrame instead of writing per event

How it works #

Two clocks, two authorities #

The transfer clock ticks as request-body bytes leave the browser. xhr.upload.onprogress reports loaded and total for a single request, and with concurrent parts in flight you sum loaded across all of them against the file’s true size. The browser owns this number and it is available immediately, at no cost.

The processing clock ticks on the server — checksum verification, virus scan, thumbnail extraction, transcode. The client cannot observe any of it, so the backend must push. Server-Sent Events are the lightweight default for one-way push; a WebSocket earns its complexity only when the client must also send control frames on the same socket, a trade-off laid out in WebSockets vs SSE for upload progress.

Keeping the two clocks separate in your state, and merging them only at the render boundary, is what stops the classic bug where a user watches the bar reach 100% and then sit there for forty seconds while FFmpeg works.

What loaded actually measures #

e.loaded is not “bytes the server has received”. It is “bytes the browser has handed to the transport”. Those bytes may be sitting in a TLS record queue, in the kernel send buffer, or in a middlebox — none of which have been acknowledged by the origin, and any of which can be discarded by a reset connection.

On a desktop with a large auto-tuned socket buffer, that gap is routinely 2–8 MB. Upload a 5 MB part over a slow uplink and the browser will report loaded === total almost immediately, then the request will sit for several seconds before onload fires. This is exactly why the last few percent of a naive bar are always a lie, and why completion must be driven by the 200 response — never by loaded === total.

The event cadence the browser gives you #

The XHR specification requires progress events to be fired at most once every 50 ms per request, with an extra event guaranteed after the final byte is queued. So a single PUT produces roughly 20 ticks per second, and six concurrent parts produce around 120 — not the “thousands” folklore suggests, but still far more than a React or Svelte tree should re-render.

The cost is not the event itself; it is what you do in the handler. A single setState per tick on a component tree with a file list, thumbnails and per-part rows costs 1–3 ms of reconciliation. At 120 ticks per second that is 12–36% of the main thread spent redrawing a bar, on the same thread that is reading Blob slices and computing checksums. Coalescing to one write per animation frame caps that at 60 writes per second regardless of concurrency, and the visual result is identical because the display cannot show more than one value per frame anyway.

Where the events come from in the DOM #

Two separate EventTargets are involved and confusing them is the most common first bug. xhr.onprogress fires for the response body — for an upload that is a handful of bytes of XML and it will look permanently stuck at 0%. xhr.upload.onprogress fires for the request body, which is what you want. The upload object also emits loadstart, load, abort, error, timeout and loadend; handlers must be attached before send(), because attaching afterwards can miss the first events for a small body served from cache-warm memory.

fetch has no equivalent because there is no request-side progress event in the Fetch standard. The nearest replacement is to pass a ReadableStream body and count bytes as they pass through, which is covered in tracking upload progress with a TransformStream; that route needs HTTP/2 and duplex: "half", so XHR remains the compatible default for the byte leg.

Step-by-step implementation #

Step 1: Capture transfer progress per part with XHR #

Wrap XMLHttpRequest in a promise and emit a tick on each progress event. The index on the tick is what lets the aggregator key by part rather than accumulate blindly.

export interface ChunkTick {
  index: number;
  loaded: number;
  total: number;
}

export function putChunk(
  url: string,
  blob: Blob,
  index: number,
  onTick: (tick: ChunkTick) => void,
  signal?: AbortSignal,
): Promise<string> {
  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest();
    xhr.open("PUT", url);
    xhr.timeout = 120_000; // a stalled socket must not hold a slot forever

    // NOTE: xhr.upload, not xhr — the latter reports the response body.
    xhr.upload.onprogress = (e: ProgressEvent) => {
      if (e.lengthComputable) onTick({ index, loaded: e.loaded, total: e.total });
    };
    xhr.onload = () => {
      if (xhr.status >= 200 && xhr.status < 300) {
        // Requires Access-Control-Expose-Headers: ETag on the bucket.
        resolve(xhr.getResponseHeader("ETag") ?? "");
      } else {
        reject(new Error(`part ${index} failed: HTTP ${xhr.status}`));
      }
    };
    xhr.onerror = () => reject(new Error(`part ${index}: network error`));
    xhr.ontimeout = () => reject(new Error(`part ${index}: timed out after 120s`));
    signal?.addEventListener("abort", () => xhr.abort(), { once: true });
    xhr.send(blob);
  });
}

Expected: a 5 MB part on a 10 Mbit/s uplink emits about 80 ticks over four seconds, { index: 0, loaded: 1048576, total: 5242880 } climbing to loaded === total, then a pause of 0.5–2 s before onload. If getResponseHeader("ETag") returns null and DevTools logs Refused to get unsafe header "ETag", the bucket CORS rule is missing the expose header — see fixing CORS preflight errors on S3 uploads.

Step 2: Aggregate progress across concurrent parts #

With several parts in flight, per-request percentages are meaningless to the user. Track the latest loaded per index and keep a running sum so each tick is O(1) rather than a walk over the map.

export class ProgressAggregator {
  private readonly loaded = new Map<number, number>();
  private sum = 0;

  constructor(private readonly fileSize: number) {}

  /** Idempotent per index: a repeated tick replaces, never adds. */
  update(tick: ChunkTick): number {
    const previous = this.loaded.get(tick.index) ?? 0;
    this.sum += tick.loaded - previous;
    this.loaded.set(tick.index, tick.loaded);
    return this.fraction;
  }

  /** Call from the retry path before a part is re-sent from byte zero. */
  reset(index: number): number {
    this.sum -= this.loaded.get(index) ?? 0;
    this.loaded.set(index, 0);
    return this.fraction;
  }

  get fraction(): number {
    return Math.min(1, Math.max(0, this.sum / this.fileSize));
  }
}

Expected: with a 20 MB file and four 5 MB parts at loaded of 5 MB, 5 MB, 2.5 MB and 0, fraction is 0.625. Calling reset(2) after a failed part drops it to 0.5 — the bar dips once, honestly, instead of silently double-counting the resend. Pair this with the retry schedule in implementing exponential backoff for failed chunks.

Step 3: Coalesce UI writes to one per frame #

Store the latest value, schedule a single flush, and let intermediate values be overwritten harmlessly. Returning a cancel function matters: a pending frame that fires after the component unmounts will write to a detached node.

export interface RafWriter {
  (value: number): void;
  cancel(): void;
}

export function rafThrottle(write: (value: number) => void): RafWriter {
  let pending: number | null = null;
  let handle = 0;

  const push = ((value: number) => {
    pending = value;
    if (handle !== 0) return;
    handle = requestAnimationFrame(() => {
      handle = 0;
      const next = pending;
      pending = null;
      if (next !== null) write(next);
    });
  }) as RafWriter;

  push.cancel = () => {
    if (handle !== 0) cancelAnimationFrame(handle);
    handle = 0;
    pending = null;
  };

  return push;
}

Expected: 2,000 calls inside one frame produce exactly one write, carrying the most recent value. In a background tab requestAnimationFrame stops firing entirely, so the queued value simply lands when the tab is foregrounded again — which is the correct behaviour, since nobody is looking.

Step 4: Receive processing progress over SSE #

Once the bytes are stored, the server keeps working. EventSource handles framing, reconnection and Last-Event-ID replay for you; your job is to map named events onto the same bar and to close the stream on a terminal state, or the browser will reconnect forever against a finished job.

export type ProcessingEvent =
  | { phase: "scan" | "thumbnail" | "transcode"; fraction: number }
  | { phase: "done" }
  | { phase: "failed"; message: string };

export function subscribeProcessing(
  url: string,
  onEvent: (event: ProcessingEvent) => void,
): () => void {
  const es = new EventSource(url, { withCredentials: true });

  es.addEventListener("progress", (e) => {
    const data = JSON.parse((e as MessageEvent<string>).data) as ProcessingEvent;
    onEvent(data);
  });
  es.addEventListener("done", () => {
    onEvent({ phase: "done" });
    es.close(); // terminal: stop the automatic reconnect loop
  });
  es.addEventListener("failed", (e) => {
    const { message } = JSON.parse((e as MessageEvent<string>).data) as { message: string };
    onEvent({ phase: "failed", message });
    es.close();
  });
  es.onerror = () => {
    // readyState 0 = reconnecting, 2 = closed for good.
    if (es.readyState === EventSource.CLOSED) {
      onEvent({ phase: "failed", message: "progress stream closed" });
    }
  };

  return () => es.close();
}

Expected: a server writing event: progress followed by data: {"phase":"transcode","fraction":0.4} advances the processing clock to 40%; event: done closes the stream. If the console shows EventSource's response has a MIME type ("text/plain") that is not "text/event-stream". Aborting the connection., the handler is missing Content-Type: text/event-stream. The jobs producing these events are typically the ones described in queueing transcode jobs with SQS and Lambda.

Step 5: Combine the two clocks into one monotonic value #

Users see one bar. Give the transfer clock a fixed share of it, give the processing clock the rest, and pass the result through a monotonic guard so no late tick can ever drag the number backwards.

export class CombinedProgress {
  private transfer = 0;
  private processing = 0;
  private shown = 0;

  constructor(private readonly transferWeight = 0.7) {}

  setTransfer(fraction: number): number {
    this.transfer = clamp01(fraction);
    return this.value();
  }

  setProcessing(fraction: number): number {
    this.processing = clamp01(fraction);
    return this.value();
  }

  /** Only call this on a deliberate restart — it is the one legal way back. */
  rewind(): number {
    this.transfer = 0;
    this.processing = 0;
    this.shown = 0;
    return 0;
  }

  private value(): number {
    const combined =
      this.transfer * this.transferWeight +
      this.processing * (1 - this.transferWeight);
    this.shown = Math.max(this.shown, combined);
    return this.shown;
  }
}

function clamp01(n: number): number {
  return Number.isFinite(n) ? Math.min(1, Math.max(0, n)) : 0;
}

Expected: a completed transfer with no processing yet reads 0.7; the transcode then fills the remaining 0.3 to 1.0. A stale tick arriving after a retry cannot lower the displayed number, and NaN from a malformed event becomes 0 instead of poisoning the bar into width: NaN%.

Step 6: Write to the DOM accessibly #

The final consumer should be a plain function that touches the DOM once per call. Use transform: scaleX() rather than width so the browser can composite the change without a layout pass, and quantise the announced value so a screen reader is not read 60 updates per second.

export interface BarParts {
  root: HTMLElement;   // gets role="progressbar"
  fill: HTMLElement;   // transform-origin: left
  status: HTMLElement; // aria-live="polite"
}

export function createBarWriter(parts: BarParts): (fraction: number) => void {
  parts.root.setAttribute("role", "progressbar");
  parts.root.setAttribute("aria-valuemin", "0");
  parts.root.setAttribute("aria-valuemax", "100");
  parts.fill.style.transformOrigin = "left";
  let announcedBucket = -1;

  return (fraction: number) => {
    const percent = Math.round(fraction * 100);
    parts.fill.style.transform = `scaleX(${fraction})`;
    parts.root.setAttribute("aria-valuenow", String(percent));

    const bucket = Math.floor(percent / 10) * 10;
    if (bucket > announcedBucket) {
      announcedBucket = bucket;
      parts.status.textContent = `${bucket}% uploaded`;
    }
  };
}

Expected: a full upload produces exactly ten polite announcements — 0%, 10%, … 90% — plus whatever terminal message you set on completion, instead of the several hundred a naive aria-valuenow-only implementation generates.

Step 7: Drive the whole upload from one loop #

The pieces compose into a small orchestrator: a fixed worker pool pulling part indices, every tick flowing through the aggregator into the combined value, and one throttled writer at the end.

export async function uploadWithProgress(
  file: File,
  partUrls: readonly string[],
  partSize: number,
  writeBar: (value: number) => void,
): Promise<string[]> {
  const aggregator = new ProgressAggregator(file.size);
  const combined = new CombinedProgress(0.7);
  const throttled = rafThrottle(writeBar);
  const etags = new Array<string>(partUrls.length).fill("");
  let nextIndex = 0;

  const worker = async (): Promise<void> => {
    for (let index = nextIndex++; index < partUrls.length; index = nextIndex++) {
      const start = index * partSize;
      const blob = file.slice(start, Math.min(start + partSize, file.size));
      etags[index] = await putChunk(partUrls[index], blob, index, (tick) => {
        throttled(combined.setTransfer(aggregator.update(tick)));
      });
    }
  };

  try {
    await Promise.all([worker(), worker(), worker(), worker()]);
  } finally {
    throttled.cancel();
  }
  writeBar(combined.setTransfer(1)); // final exact value, unthrottled
  return etags;
}

Expected: for a 200 MB file at 8 MB parts, four workers keep 32 MB in flight, the bar advances continuously rather than in 25-point jumps, and the returned etags array is ready for the completion call. Choosing the part size and concurrency is its own trade-off, discussed in multipart vs single-PUT for files under 100MB.

Configuration reference #

Budgeting the bar across processing phases #

transferWeight = 0.7 is a starting point, not a law. The right split is the measured ratio of median transfer time to median processing time for your workload. Instrument both, take the p50 over a week, and set the weight to transferSeconds / (transferSeconds + processingSeconds). For a photo pipeline that resizes a 4 MB JPEG in 300 ms, transfer dominates and the weight belongs near 0.95. For a 200 MB ProRes clip that takes four minutes to transcode, transfer is a third of the wall clock and 0.35 is closer to honest.

Inside the processing share, give each phase its own sub-budget rather than letting the backend emit one undifferentiated fraction. A phase-weighted map keeps the bar moving smoothly even when one stage reports coarsely:

const PHASE_WEIGHTS = { scan: 0.2, thumbnail: 0.3, transcode: 0.5 } as const;
const PHASE_ORDER = ["scan", "thumbnail", "transcode"] as const;

export function processingFraction(
  phase: (typeof PHASE_ORDER)[number],
  within: number,
): number {
  let completed = 0;
  for (const name of PHASE_ORDER) {
    if (name === phase) break;
    completed += PHASE_WEIGHTS[name];
  }
  return completed + PHASE_WEIGHTS[phase] * Math.min(1, Math.max(0, within));
}

Expected: processingFraction("transcode", 0.5) returns 0.75, which the CombinedProgress with weight 0.7 renders as 0.925 on the visible bar. A scan that only ever reports 0 then 1 still contributes a clean 20% step rather than a stall.

When several files upload at once, weight each file’s contribution to the queue bar by its byte size, not by file count: ten 40 KB thumbnails plus one 2 GB video is 99.8% one file, and a bar that treats them as eleven equal units will sit at 91% for twenty minutes. Preprocessing that shrinks files before they are queued — see client-side media preprocessing — changes those weights, so compute them after preprocessing, never from the original File.size.

Edge cases and gotchas #

lengthComputable is false #

Behind some proxies, and whenever the body is sent with chunked transfer encoding and no Content-Length, e.lengthComputable is false and e.total is 0. Dividing by it yields Infinity, and scaleX(Infinity) silently renders nothing. Guard the handler as in step 1, keep that part out of the sum, and scale the denominator to the parts that do report — or fall back to an indeterminate animation for the whole transfer if none of them do.

The plateau at the end of every part #

Because loaded counts handoff to the socket, each part’s ticks finish well before its 200 arrives. With four concurrent parts you get four little plateaus, and the final one is the longest because there is no other part left to advance the sum. Two fixes work together: reserve the processing share of the bar so there is always something left to fill, and cap the transfer clock at 0.99 until every ETag is in hand.

Retry storms double-count bytes #

A part that fails at 4 MB of 5 MB and restarts will emit ticks from zero again. Without reset(index) the running sum keeps the abandoned 4 MB and the bar drifts above the real figure, then hits 100% while two parts are still uploading. Always call reset in the catch before re-dispatching, and keep the retry count bounded — resuming uploads after network loss covers the surrounding recovery logic.

The six-connection cap per origin #

Browsers allow around six concurrent HTTP/1.1 connections per origin, and every EventSource holds one for the lifetime of the job. Four upload workers plus one progress stream plus a polling call is already at the ceiling, and the seventh request queues invisibly — it looks like a slow server, not a client-side stall. Open exactly one processing stream per upload session, put it on the same origin as an HTTP/2 endpoint if you can, and never open one per part.

Buffering proxies hold events hostage #

Nginx buffers proxied responses by default, so SSE frames accumulate until the buffer fills or the connection closes; the client sees nothing for minutes and then eight events at once. Send X-Accel-Buffering: no from the handler, or set proxy_buffering off for that location. The symptom is distinctive: a curl -N against the origin streams correctly while the same request through the edge does not.

Credentialed streams and wildcard CORS #

new EventSource(url, { withCredentials: true }) fails if the response carries Access-Control-Allow-Origin: *, with The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'. Echo the exact origin and add Access-Control-Allow-Credentials: true.

Cancellation must reset both clocks #

When a user cancels, aborting the in-flight XHRs is not enough — the SSE subscription stays open and can push a done event for a job you no longer care about, snapping a cancelled bar to 100%. Tear down both, and drop the aggregator instance rather than reusing it. The signal plumbing is covered in aborting uploads with AbortController and timeouts.

Percentages are not time #

A bar at 60% says nothing about how long the remaining 40% will take, because the transfer and processing shares run at different rates. If you show a countdown alongside the bar, derive it from measured throughput rather than from the bar value — the estimator in showing accurate time-remaining estimates exists precisely because the naive division is wrong in a reproducible way.

Verification #

Start with the raw event stream. curl -N disables output buffering, so frames appear as the server writes them; if they arrive in one burst at the end, the problem is buffering, not your client.

# Frames should appear one at a time, seconds apart.
curl -N -H 'Accept: text/event-stream' \
  https://api.example.com/uploads/abc123/events
# event: progress
# data: {"phase":"scan","fraction":0.4}
#
# event: progress
# data: {"phase":"transcode","fraction":0.9}
#
# event: done
# data: {}

Then assert the pure logic without a network. Every class above is deterministic, so the whole progress model is testable in a few lines under node --test:

import { test } from "node:test";
import assert from "node:assert/strict";
import { ProgressAggregator } from "./aggregator.js";
import { CombinedProgress } from "./combined.js";

test("aggregate is idempotent per part index", () => {
  const aggregator = new ProgressAggregator(20 * 1024 * 1024);
  aggregator.update({ index: 0, loaded: 5 * 1024 * 1024, total: 5 * 1024 * 1024 });
  aggregator.update({ index: 0, loaded: 5 * 1024 * 1024, total: 5 * 1024 * 1024 });
  aggregator.update({ index: 1, loaded: 5 * 1024 * 1024, total: 5 * 1024 * 1024 });
  assert.equal(aggregator.fraction, 0.5);
});

test("a retry rewinds only its own part", () => {
  const aggregator = new ProgressAggregator(10 * 1024 * 1024);
  aggregator.update({ index: 0, loaded: 5 * 1024 * 1024, total: 5 * 1024 * 1024 });
  aggregator.update({ index: 1, loaded: 4 * 1024 * 1024, total: 5 * 1024 * 1024 });
  assert.equal(aggregator.reset(1), 0.5);
});

test("the displayed value never regresses", () => {
  const combined = new CombinedProgress(0.7);
  assert.equal(combined.setTransfer(1), 0.7);
  assert.equal(combined.setTransfer(0.4), 0.7);
  assert.equal(combined.setProcessing(1), 1);
});

In the browser, three DevTools checks catch the remaining problems. Throttle the network to “Slow 4G” and watch the Network panel: the request rows should stay open for seconds after the bar reaches its transfer cap, which confirms the socket-buffer gap is real and that your cap is doing its job. Record a Performance profile during an upload and count the style-recalculation entries — with coalescing you should see one per frame, not one per tick. Finally, open the EventStream tab on the SSE request; each named event should be listed with its own timestamp, and the gaps between them should match the backend’s real work, not arrive as one burst of rows at the end.

Frequently Asked Questions