Reacting to Azure Blob Uploads with Event Grid #

Create an Event Grid system topic for the storage account, add an event subscription filtered to Microsoft.Storage.BlobCreated with a subjectBeginsWith of /blobServices/default/containers/uploads/blobs/originals/, deliver to a Storage Queue (or Service Bus queue) with a dead-letter container configured, and in the consumer skip events whose api is PutBlock or whose data.contentLength is zero, deduplicating on the blob URL plus eTag.

Azure Blob Storage publishes object events through Event Grid, and the details trip people up in predictable ways: block blob uploads can produce events at different moments depending on the API used, subject filters use a path format that is not the blob URL, and the default delivery retries for a full day, so a broken consumer quietly accumulates a backlog. This page belongs to upload completion events in backend validation and cloud storage architecture. The upload side is covered in uploading to Azure Blob with the Storage JS SDK.

When to use this approach #

• Browsers upload directly to Azure Blob Storage with SAS tokens, and processing must start when each upload is committed.
• Several services — scanning, thumbnails, indexing — react to the same uploads independently.
• You want managed delivery with retries and dead-lettering rather than polling containers.

Prerequisites #

1. A StorageV2 (general-purpose v2) account; Event Grid does not publish events for legacy account kinds.
2. The Microsoft.EventGrid resource provider registered in the subscription.
3. A destination: a Storage Queue, Service Bus queue, Event Hub, Azure Function or webhook. The examples use a Storage Queue.
4. A blob container for dead-lettered events, and @azure/storage-queue 12.x for the consumer.

How an upload becomes an event #

Implementation #

Create the system topic and subscription with the Azure CLI:

RG=media-rg; ACCOUNT=mediauploads; QUEUE=uploads-to-processor
ACCOUNT_ID=$(az storage account show -g $RG -n $ACCOUNT --query id -o tsv)

az eventgrid system-topic create -g $RG --name ${ACCOUNT}-events \
  --location westeurope --topic-type Microsoft.Storage.StorageAccounts --source "$ACCOUNT_ID"

az eventgrid system-topic event-subscription create -g $RG \
  --system-topic-name ${ACCOUNT}-events --name to-processor \
  --included-event-types Microsoft.Storage.BlobCreated \
  --subject-begins-with /blobServices/default/containers/uploads/blobs/originals/ \
  --advanced-filter data.api StringIn PutBlob PutBlockList CopyBlob FlushWithClose \
  --advanced-filter data.contentLength NumberGreaterThan 0 \
  --endpoint-type storagequeue --endpoint "$ACCOUNT_ID/queueservices/default/queues/$QUEUE" \
  --max-delivery-attempts 10 --event-ttl 1440 \
  --deadletter-endpoint "$ACCOUNT_ID/blobServices/default/containers/eventgrid-deadletter"

The consumer, reading Event Grid events from the Storage Queue:

import { QueueClient } from "@azure/storage-queue";
import { DefaultAzureCredential } from "@azure/identity";

const queue = new QueueClient(
  `https://mediauploads.queue.core.windows.net/uploads-to-processor`,
  new DefaultAzureCredential(),
);

interface BlobCreatedEvent {
  id: string;
  subject: string;                      // /blobServices/default/containers/uploads/blobs/originals/9c1f/a.jpg
  eventType: "Microsoft.Storage.BlobCreated";
  eventTime: string;
  data: {
    api: "PutBlob" | "PutBlockList" | "CopyBlob" | "FlushWithClose";
    url: string;
    eTag: string;
    contentType: string;
    contentLength: number;
    blobType: "BlockBlob" | "AppendBlob" | "PageBlob";
    sequencer: string;
  };
}

const done = new Set<string>();                         // replace with a durable store

export async function drain(): Promise<void> {
  for (;;) {
    const { receivedMessageItems } = await queue.receiveMessages({ numberOfMessages: 16, visibilityTimeout: 300 });
    if (receivedMessageItems.length === 0) { await new Promise((r) => setTimeout(r, 2000)); continue; }
    for (const m of receivedMessageItems) {
      // Event Grid writes the event as base64-encoded JSON into the message body.
      const evt = JSON.parse(Buffer.from(m.messageText, "base64").toString("utf8")) as BlobCreatedEvent;
      const key = `${evt.data.url}#${evt.data.eTag}`;
      try {
        if (!done.has(key)) {
          const blob = decodeURIComponent(new URL(evt.data.url).pathname.split("/").slice(2).join("/"));
          console.log(JSON.stringify({ msg: "new blob", blob, bytes: evt.data.contentLength, api: evt.data.api }));
          // process here
          done.add(key);
        }
        await queue.deleteMessage(m.messageId, m.popReceipt);
      } catch (err) {
        if (m.dequeueCount >= 5) {
          console.error(JSON.stringify({ msg: "giving up", key, err: String(err) }));
          await queue.deleteMessage(m.messageId, m.popReceipt);   // or move to a poison queue
        }
        // otherwise leave it: it becomes visible again after the visibility timeout
      }
    }
  }
}

await drain();

Line-by-line on the settings that matter #

• The subject format. Subjects look like /blobServices/default/containers/<container>/blobs/<path>. A prefix filter on the plain blob URL or on uploads/originals/ matches nothing. Get one real event, copy its subject, and cut it where you want the filter.
• data.api filter. PutBlockList is the commit of a block upload (what the JS SDK’s uploadData and uploadBrowserData do for large files); PutBlob is a single-shot upload; FlushWithClose is the Data Lake Gen2 equivalent. Filtering on these excludes noise such as CopyBlob from your own processing if you remove it from the list.
• data.contentLength > 0. Some tools create a zero-length placeholder before writing content. Filtering them out avoids processing empty blobs.
• --max-delivery-attempts 10 --event-ttl 1440. Event Grid retries with exponential backoff until either limit is reached, then writes the event to the dead-letter container. The defaults (30 attempts, 24 hours) are generous; lower them so failures surface sooner.
• Base64 message body. Event Grid’s Storage Queue delivery writes each event as base64-encoded JSON. Decoding the body directly as JSON fails with Unexpected token.
• dequeueCount. Storage Queues have no built-in dead-letter queue. Checking dequeueCount and moving or deleting after a limit gives you the same protection, as described in handling poison messages with dead-letter queues.

Which API calls produce events #

Choosing the delivery destination #

Event Grid can deliver to several endpoint types, and the choice decides how failures behave. Storage Queues are cheap and simple, with no built-in dead-lettering and a 64 KB message limit — fine for Event Grid events, which are small. Service Bus queues add sessions, duplicate detection within a window, native dead-letter queues and larger messages; choose them when consumers need those guarantees. Azure Functions with an Event Grid trigger are the least code, but retries then happen at two layers (Event Grid redelivery and the function runtime), which can surprise you with more duplicate executions than expected. Webhooks need an endpoint that completes Event Grid’s validation handshake and responds quickly; long processing behind a webhook leads to timeouts and redelivery.

For media pipelines, a queue between Event Grid and the worker is almost always right: it absorbs bursts after a bulk upload, lets you see the backlog, and lets workers take minutes per item without Event Grid treating that as a failed delivery.

Monitoring the subscription #

Event Grid exposes per-subscription metrics that answer “are uploads being processed?” without looking at the worker. Watch Delivery Failed Events and Dead Lettered Events — both should be zero in steady state — and Delivery Attempt Fail Count broken down by error, which distinguishes a missing queue from an authentication problem. On the queue, alert on the approximate message count and on the age of the oldest message; a growing count with a healthy Event Grid means the worker has stopped. Periodically list the dead-letter container: each blob there is an event that never reached your consumer, with the failure reason recorded alongside it.

Configuration gotchas #

Microsoft.EventGrid resource provider is not registered. Register it once per subscription: az provider register --namespace Microsoft.EventGrid.

No events for uploads that clearly succeeded. The account is a legacy kind (Storage or BlobStorage) rather than StorageV2, or the subject filter does not match because it omits /blobServices/default/containers/. Test with a filter-free subscription first.

Events for every block of a large upload. They are not BlobCreated events — you subscribed to all event types including Data Lake FlushWithClose on append flows, or the uploader commits frequently with Put Block List. Check data.api in a captured event.

Consumer cannot parse messages. Messages are base64 JSON. Decode first; the SDK’s messageEncoding option does not apply to Event Grid-written messages.

Verification #

# Capture one real event: temporarily add a Storage Queue subscription without filters and peek.
az storage message peek --account-name mediauploads --queue-name uploads-to-processor --auth-mode login \
  --query '[0].content' -o tsv | base64 -d | jq '{subject, api: .data.api, len: .data.contentLength}'

# Upload a test blob and confirm one message arrives.
az storage blob upload --account-name mediauploads --container-name uploads \
  --name originals/test/photo.jpg --file photo.jpg --auth-mode login
az storage message peek --account-name mediauploads --queue-name uploads-to-processor --auth-mode login --num-messages 5

Frequently Asked Questions