Streaming Upload Progress with Server-Sent Events #

Server-Sent Events push server-side processing progress to the browser over one long-lived HTTP response, and EventSource reconnects on its own with Last-Event-ID so the UI never goes stale.

The bytes-uploaded number from an XMLHttpRequest only tells half the story. Once the last byte lands, the backend still scans, transcodes, and indexes the file — work the browser cannot observe. Streaming that server-authoritative progress is a core concern of realtime upload progress events within frontend UX, chunking and progress tracking. SSE is the lightest fit: one GET returns text/event-stream, the server appends data: lines as work proceeds, and EventSource handles framing, event dispatch and reconnection for you. The events themselves usually originate in post-upload media transcoding or automated virus scanning.

When to use this approach #

• Updates are one-directional. The server reports; the client never needs to push a message back on the same channel. If it does, read WebSockets vs SSE for upload progress before committing.
• You want automatic reconnection and event replay without writing a heartbeat, a backoff loop, or a resume cursor by hand.
• Payloads are small JSON blobs at a handful of events per second. SSE is UTF-8 text only; binary frames mean base64 and a 33% overhead.

Prerequisites #

1. Node 20+ (or any runtime that can hold a response open and flush incrementally) and a bundler that emits ESM.
2. A job identifier minted when the upload starts, so the stream and the upload refer to the same work item.
3. A reverse proxy you can configure — buffering and idle timeouts are the two things that break SSE in production.
4. CORS configured if the stream is cross-origin; see CORS configuration for uploads.

How the wire format actually works #

SSE is not a framing protocol in the WebSocket sense. It is a plain text body parsed line by line. The browser reads UTF-8 lines separated by \n, \r\n or a bare \r, splits each on the first colon, and accumulates fields into a buffer. A blank line dispatches whatever has accumulated; anything still in the buffer when the connection drops is discarded.

Only four field names mean anything — event, data, id, retry. Every other name is silently ignored, which is what makes a line beginning with a colon a comment. A single space after the colon is stripped, so data: {"a":1} and data:{"a":1} are identical. Multiple data: lines within one event are joined with a newline, and the trailing newline is removed before dispatch. Crucially, an event with an empty data buffer is not dispatched at all: event: ping on its own fires nothing, which is exactly why keep-alives are written as comments rather than as named events.

Implementation #

The server #

The server holds the response open, replays anything the client missed, and writes a comment every 15 seconds so no intermediary decides the connection is idle. This example keeps a bounded in-memory history per job; in a multi-process deployment, back it with Redis or read it from the job row instead.

// sse-server.ts — Node 20+, ESM. Run with: node --experimental-strip-types sse-server.ts
import { createServer } from "node:http";
import { EventEmitter } from "node:events";

export interface JobEvent {
  id: number;
  name: "progress" | "done" | "error";
  data: unknown;
}

const KEEPALIVE_MS = 15_000; // shorter than any proxy read timeout on the path
const RETRY_MS = 5_000;      // client-side reconnect delay
const REPLAY_LIMIT = 100;    // per-job ring buffer depth

const bus = new EventEmitter();
const history = new Map<string, JobEvent[]>();
let seq = 0;

/** Call this from the scanner, transcoder or indexer as each stage advances. */
export function publish(jobId: string, name: JobEvent["name"], data: unknown): void {
  const event: JobEvent = { id: ++seq, name, data };
  const log = history.get(jobId) ?? [];
  log.push(event);
  if (log.length > REPLAY_LIMIT) log.shift();
  history.set(jobId, log);
  bus.emit(jobId, event);
}

function frame(event: JobEvent): string {
  // JSON.stringify never emits a raw newline, so one data line is always enough.
  return `id: ${event.id}\nevent: ${event.name}\ndata: ${JSON.stringify(event.data)}\n\n`;
}

const server = createServer((req, res) => {
  const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`);
  const match = /^\/api\/jobs\/([\w-]+)\/events$/.exec(url.pathname);
  if (!match) {
    res.writeHead(404).end();
    return;
  }
  const jobId = match[1];

  res.writeHead(200, {
    "Content-Type": "text/event-stream; charset=utf-8",
    "Cache-Control": "no-cache, no-transform", // no-transform blocks proxy re-compression
    "Connection": "keep-alive",                // ignored, and harmless, on HTTP/2
    "X-Accel-Buffering": "no",                 // tells Nginx not to buffer the body
  });
  res.flushHeaders();
  res.socket?.setNoDelay(true); // Nagle would sit on 80-byte frames for 40ms
  res.socket?.setTimeout(0);    // never idle-close the response socket

  res.write(`retry: ${RETRY_MS}\n\n`);

  // Replay whatever this client missed. EventSource cannot set headers, so accept
  // a query-string fallback for polyfills and for curl-based debugging.
  const lastSeen = Number(
    req.headers["last-event-id"] ?? url.searchParams.get("lastEventId") ?? 0,
  );
  for (const past of history.get(jobId) ?? []) {
    if (past.id > lastSeen) res.write(frame(past));
  }

  const onEvent = (event: JobEvent) => {
    res.write(frame(event));
  };
  bus.on(jobId, onEvent);

  const keepAlive = setInterval(() => {
    res.write(": keep-alive\n\n");
  }, KEEPALIVE_MS);

  const cleanup = () => {
    clearInterval(keepAlive);
    bus.off(jobId, onEvent);
  };
  res.on("close", cleanup);
  res.on("error", cleanup);
});

server.requestTimeout = 0; // Node 18+ defaults to 300_000 and would cut long streams
server.headersTimeout = 0;
server.listen(8080);

The client #

EventSource parses the stream and dispatches by event name. The code below adds the one thing the browser will not do for you: recover from a fatal handshake failure, which never retries automatically.

export interface ProgressPayload {
  jobId: string;
  stage: "scanning" | "transcoding" | "storing";
  percent: number;
}

export interface ProgressHandlers {
  onProgress: (p: ProgressPayload) => void;
  onDone: (jobId: string) => void;
  onFailure: (message: string) => void;
}

const MAX_FATAL_RETRIES = 4;

export function trackProcessing(jobId: string, handlers: ProgressHandlers): () => void {
  const url = `/api/jobs/${encodeURIComponent(jobId)}/events`;
  let source: EventSource | null = null;
  let fatalRetries = 0;
  let reopenTimer: ReturnType<typeof setTimeout> | undefined;
  let disposed = false;

  const dispose = () => {
    disposed = true;
    clearTimeout(reopenTimer);
    source?.close();
    source = null;
  };

  const open = () => {
    source = new EventSource(url, { withCredentials: true });

    source.addEventListener("open", () => {
      fatalRetries = 0; // a successful handshake resets the budget
    });

    source.addEventListener("progress", (ev) => {
      handlers.onProgress(JSON.parse((ev as MessageEvent).data) as ProgressPayload);
    });

    source.addEventListener("done", (ev) => {
      const { jobId: finished } = JSON.parse((ev as MessageEvent).data) as { jobId: string };
      handlers.onDone(finished);
      dispose(); // terminal — without this the browser reconnects and replays
    });

    source.addEventListener("error", (ev) => {
      const raw = (ev as MessageEvent).data;
      if (typeof raw === "string" && raw.length > 0) {
        // Application failure: the server sent `event: error` with a JSON body.
        handlers.onFailure((JSON.parse(raw) as { message: string }).message);
        dispose();
        return;
      }
      // Transport failure. CONNECTING means the browser is already retrying itself.
      if (!source || source.readyState === EventSource.CONNECTING) return;
      source.close();
      if (disposed) return;
      if (fatalRetries >= MAX_FATAL_RETRIES) {
        handlers.onFailure("Progress stream unavailable after 4 attempts.");
        return;
      }
      const backoff = Math.min(30_000, 1000 * 2 ** fatalRetries);
      fatalRetries += 1;
      reopenTimer = setTimeout(open, backoff * (0.5 + Math.random() / 2));
    });
  };

  open();
  return dispose;
}

// --- Usage ---
const bar = document.querySelector<HTMLProgressElement>("#bar")!;
const label = document.querySelector<HTMLSpanElement>("#stage")!;

const stop = trackProcessing("job_8f3a", {
  onProgress: ({ stage, percent }) => {
    bar.value = percent;
    label.textContent = stage;
  },
  onDone: (id) => {
    label.textContent = "complete";
    console.log("[sse] finished", id);
  },
  onFailure: (msg) => console.error("[sse]", msg),
});

window.addEventListener("beforeunload", stop);

Line-by-line of the critical parts #

• res.flushHeaders() sends the status line and headers before the first event. Without it Node waits for the first write(), and a client that opens the stream before any work starts sits in CONNECTING with no open event.
• retry: 5000 as the first frame sets the reconnection delay for the lifetime of that EventSource. The browser default is implementation-defined (roughly 3 seconds in Chrome and Firefox); state it explicitly so your reconnect load is predictable.
• The id: on every frame is what the browser echoes as Last-Event-ID. Use a monotonic integer, not a UUID — the server has to answer “everything after this” cheaply.
• ": keep-alive\n\n" is a comment with no data buffer, so no listener fires. It exists purely to put bytes on the socket.
• new EventSource(url, { withCredentials: true }) issues a GET with Accept: text/event-stream and sends cookies cross-origin. There is no options object for headers — that limitation drives several decisions below.
• addEventListener("progress", …) matches the server’s event: progress field. Omit the event: line and everything arrives on message instead.
• dispose() on done is mandatory. SSE has no server-initiated close that stops the client; if the server simply ends the response, the browser reconnects after retry ms and the server replays from Last-Event-ID, firing done a second time.

The sequence below shows the whole lifecycle, including the drop that the browser repairs without your code being involved.

Configuration reference #

Telling a retry apart from a fatal failure #

This is the part that catches teams out. A mid-stream drop and a bad handshake both fire an error event, but the browser treats them completely differently. If the response arrives with a status other than 200, or a Content-Type that is not text/event-stream, or the TCP connection cannot be established at all, the browser fails the connection: it fires error once, sets readyState to 2, and never retries. A 204 No Content reply on reconnect does the same thing deliberately — it is the documented way for a server to tell a client to stop. Only a drop after a successful handshake puts the object back into CONNECTING and schedules an automatic retry.

Multiplexing many jobs onto one stream #

A gallery uploader that opens one EventSource per file hits the browser’s per-origin connection limit almost immediately. Over HTTP/1.1 that limit is six, and SSE streams never close, so the seventh file’s stream — and every ordinary fetch the page makes — queues behind them indefinitely. The symptom is maddening: uploads stall with no error anywhere, and the network panel shows requests stuck in “Queued”.

Two fixes exist. Serve the endpoint over HTTP/2, where a single TCP connection multiplexes roughly 100 concurrent streams, or collapse progress onto one stream and put the job identifier inside each frame. The second is worth doing regardless: it costs one server-side subscription instead of n, and it survives a client that falls back to HTTP/1.1 through a corporate proxy.

Configuration gotchas #

Events arrive in one burst at the end #

The job finishes and every frame lands at once. An intermediary is buffering the response body. Nginx buffers by default: set proxy_buffering off; for the location, or emit X-Accel-Buffering: no as above, and add proxy_http_version 1.1; plus gzip off; for that route — gzip accumulates a compression window before flushing anything. The same class of proxy setting bites during upload; the surrounding configuration is covered in raising Nginx and Cloudflare upload size limits.

EventSource’s error fires once and the UI freezes #

The console shows EventSource's response has a MIME type ("text/plain") that is not "text/event-stream". Aborting the connection. — usually an error page from a proxy or a framework’s default JSON error handler. Because this is a fatal failure the browser will not retry, which is why the client above tracks fatalRetries and reopens itself. Never return a 500 body on this route; report application failures as an in-band event: error frame on a 200 response instead.

The stream dies at exactly 60 seconds #

Nginx’s proxy_read_timeout defaults to 60s, AWS ALB’s idle timeout to 60s, and Cloudflare’s proxy closes at around 100s of silence. A job that spends two minutes in a transcode queue emits nothing, so the connection is torn down and the client reconnects on a loop. The 15-second keep-alive comment fixes it for every intermediary at once. Raise proxy_read_timeout 3600s; as belt and braces, but do not rely on it alone.

CORS rejects the stream even though the endpoint works #

With withCredentials: true, a wildcard is invalid and you get The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'. Echo the exact request origin, send Access-Control-Allow-Credentials: true, and remember that EventSource sends a simple GET — there is no preflight to debug, so a missing header shows up only on the real request.

Every client reconnects at the same instant after a deploy #

Rolling a new revision drops every open stream simultaneously, and a fixed retry: 5000 brings them all back in the same 100 ms window. Jitter the reopen on the client, as the implementation above does, and cap concurrent stream acceptances server-side. The same reasoning as exponential backoff for failed chunks applies here.

Verification #

Prove the framing and the replay from a terminal before wiring any UI. curl -N disables output buffering so you see frames as they arrive.

# 1. Headers only — Content-Type is the single most common cause of a dead stream.
curl -sSI https://api.example.com/api/jobs/job_8f3a/events | grep -Ei 'content-type|cache-control|accel'
# content-type: text/event-stream; charset=utf-8
# cache-control: no-cache, no-transform
# x-accel-buffering: no

# 2. Watch raw frames. Each block must end with a blank line.
curl -N -H "Accept: text/event-stream" https://api.example.com/api/jobs/job_8f3a/events
# retry: 5000
#
# id: 1
# event: progress
# data: {"jobId":"job_8f3a","stage":"scanning","percent":30}
#

# 3. Prove replay: ask for everything after event 1 and confirm id 1 is not resent.
curl -N -H "Accept: text/event-stream" -H "Last-Event-ID: 1" \
  https://api.example.com/api/jobs/job_8f3a/events

# 4. Prove incremental flushing: timestamps must be seconds apart, not identical.
curl -N -s https://api.example.com/api/jobs/job_8f3a/events | while read -r line; do
  printf '%s %s\n' "$(date +%T)" "$line"
done

In DevTools, open the request and use the EventStream tab: Chrome lists each dispatched event with its id, type and data, which is the fastest way to confirm that a frame you think you sent was actually parsed rather than swallowed as an unknown field. Once the stream is trustworthy, feed its percent values into the smoothing described in showing accurate time-remaining estimates rather than binding them straight to a bar.

Frequently Asked Questions