Six Real-Time Patterns in Azure: What the Diagram Leaves Out

Posted on October 1, 2026 by steefjan1970

I see a diagram of real-time communication patterns pop up in my feed every few weeks. Polling, long polling, Server-Sent Events, WebSockets, webhooks, gRPC streaming. Six neat boxes, and one piece of sensible advice underneath: choose the simplest model that fits the use case.

The diagram is correct. However, it is not the part that costs you a sprint.

Picking a pattern takes about five minutes. Making that pattern survive API Management, Azure Front Door, a scale-out event, and a rolling deployment takes considerably longer. This post covers that second part. It also comes with a working sample that runs all six patterns from one image, against one shared event source. Azure needs two apps to host it, and that’s one of the more interesting findings below.

Four questions, not six boxes

A grid of six options invites you to shop. Instead, answer four questions in order. The pattern then picks itself.

Which direction does data flow? The client pulls, the server pushes, or both sides talk at once. This single question removes at least three options.

What is the latency budget? Seconds are cheap. Milliseconds are not. Most business dashboards tolerate a delay that their designers never measured.

Who owns the connection when it drops? Someone must handle reconnect, resume, and replay. If your answer is “the browser does that automatically”, read the reconnect semantics again first.

Who pays per open socket? Idle connections still consume replicas, units, and money.

When two patterns both fit, pick the one that keeps state out of the connection. A dropped request costs a retry. A dropped session costs a reconnect, a replay, and a support ticket.

The six patterns, and what bites in Azure

Polling

The client asks again on its own schedule. Azure itself uses this pattern constantly. The asynchronous request-reply pattern returns 202 Accepted with a Location header, and Durable Functions exposes a status query endpoint that works exactly this way.

What bites: the missing Retry-After header. Without it, every client invents its own interval. Ten thousand clients then converge on the same second after a deployment, and your scale rule reacts to a spike you created yourself.

Long polling

The server holds the request open until data arrives or the clock runs out. Azure Service Bus applies the same idea inside its SDK, where a receive call waits for a configurable maximum wait time instead of returning empty.

What bites: timeouts you do not control. App Service and Azure Functions enforce a fixed 230-second ceiling on HTTP requests. That number comes from the Azure Load Balancer underneath, which idles connections out at 240 seconds by default. You cannot raise it because it is a platform constraint, not an application setting. Front Door then applies its own origin response timeout on top. Therefore, derive your hold time from the shortest timeout in the path, not from the longest. The sample holds for 25 seconds, which clears every layer.

Server-Sent Events

The server pushes over one long-lived HTTP response. SSE is unfashionable and quietly excellent. It runs over plain HTTP, it survives proxies that understand chunked responses, and browsers reconnect on their own.

You already depend on it. Token streaming from Azure OpenAI and Microsoft Foundry arrives as Server-Sent Events. Every chat interface you have built this year uses this pattern, whether or not the architecture diagram says so.

What bites: buffering, which gets its own section below. Also reconnect gaps. The browser resends the Last-Event-ID header automatically, but the server must honor it. Otherwise, every reconnect silently drops the events that arrived while the socket was down.

WebSockets

Both sides talk over one persistent connection. This is the right answer for chat, collaborative editing, and live trading. It is the wrong answer for a dashboard that changes twice an hour.

Azure gives you two managed options. That is Azure Web PubSub, which handles raw WebSocket clients and works well outside .NET. And Azure SignalR Service fits when you already use hubs and want fallbacks.

What bites: the deployment. A rolling revision in Azure Container Apps drops every open socket at once. All those clients then reconnect together, which looks exactly like an attack to your scale rules. Managed services exist mainly to move that problem off your replicas.

Webhooks

One system calls another when an event happens. Azure Event Grid delivers this way, with retries, dead-lettering, and support for the CloudEvents schema.

What bites: the unglamorous eighty percent, and it starts before your first event arrives.

First, you must pass a validation handshake, and the shape depends on your schema. The native Event Grid schema sends a POST carrying a SubscriptionValidationEvent. You read validationCode from the data object and echo it back as {"validationResponse": "<code>"} with a 200, within 30 seconds. As a result, an endpoint that authenticates, queues, and logs before responding can miss that window on a cold start. The CloudEvents schema works differently. It sends an HTTP OPTIONS preflight carrying a WebHook-Request-Origin header, which you echo back as WebHook-Allowed-Origin. Afterward, every real delivery carries a matching Origin header that you can check.

Second, verify signatures over the raw body, because serialization isn’t byte-stable.

Third, deduplicate. At-least-once delivery makes duplicates a certainty rather than an edge case.

Fourth, answer 400 rather than 500 when a payload will not parse. Event Grid retries 5xx responses because a server error suggests a temporary problem. A body your parser cannot read is not temporary. So an unhandled exception turns one bad payload into a retry cycle that runs until the event dead-letters, and no attempt in that cycle can ever succeed. I introduced this exact bug in the sample while writing this post, then watched a shell-quoting mistake surface it.

Ordering bites too. Event Grid delivers at least once and never guarantees sequence. A failed event enters an exponential backoff queue, while later events sail straight past it. Retries therefore scramble order actively rather than occasionally. So build consumers around sequence numbers or timestamps in the payload. Alternatively, when order is structural rather than incidental, pull from Event Hubs or a Service Bus session instead.

gRPC streaming

Service-to-service, strongly typed, over HTTP/2. ASP.NET Core supports server, client, and bidirectional streaming out of the box.

What bites: the protocol has to match on both sides of the ingress, and the two halves are configured in different places.

Start with ingress. The transport property governs the protocol between the ingress proxy and your container, not only at the edge. Container Apps needs transport: http2 for gRPC. However, the WebSocket handshake relies on the HTTP/1.1 Upgrade mechanism, which HTTP/2 replaces with stream multiplexing. So the two protocols sit badly behind one ingress.

Now the container. Kestrel needs an explicit Protocols: Http2 setting to speak cleartext HTTP/2. The Http1AndHttp2 value looks like it covers both, yet it quietly falls back to HTTP/1.1 without TLS, because negotiation depends on ALPN. Container Apps terminates TLS at the ingress, so your container never sees a handshake to negotiate over.

Kestrel says so during startup, in plain language:

			
HTTP/2 is not enabled for [::]:8080. The endpoint is configured to use
HTTP/1.1 and HTTP/2, but TLS is not enabled. HTTP/2 requires TLS application
protocol negotiation. Connections to this endpoint will use HTTP/1.1.

That line sits in your container logs from the first boot. Nobody reads startup logs when the application starts successfully, so it waits there until a gRPC call fails hours later.

Get one half right, and the other wrong, and the client receives this instead:

			
upstream connect error or disconnect/reset before headers.
retried and the latest reset reason: remote refused stream reset

That message names the upstream connection, which sends you to inspect ingress, networking, and scale rules. Meanwhile, the container is the one answering in the wrong protocol. So read the container startup logs first when a gRPC call fails at the edge. The answer usually arrives before the question.

The sample settles both halves at once. It listens on port 8080 for HTTP, SSE, and WebSockets over HTTP/1.1, and on port 8081 for gRPC over h2c. It then deploys the same image twice, once with transport: auto pointed at 8080, and once with transport: http2 pointed at 8081. One image, two ports, two ingress configurations, no compromise.

What the edge does to your stream

Here is the failure I keep seeing, and it never appears on the pattern diagram.

You build an SSE endpoint. It works perfectly on your laptop. You then publish it through API Management, and every event stops arriving. Nothing appears for two minutes, and then the whole stream lands at once, or the connection simply times out.

Your code is fine. The gateway buffers the response by default. It collects chunks from the backend, typically in 8 KB increments, and forwards them only when the buffer fills, or the stream ends. That is reasonable behavior for a normal API and fatal for a stream.

The fix is one attribute in the backend policy:

			
<backend>
  <forward-request buffer-response="false" timeout="240" />
</backend>

Set buffer-response to false, and the gateway forwards each chunk as it arrives. Note the hyphen. An underscore looks close enough to survive a code review and fails policy validation.

Meanwhile, three more layers deserve a look before you ship:

Kestrel buffers writes unless you call DisableBuffering and flush after each event.
Front Door applies an origin response timeout. Standard and Premium default to 60 seconds, while the Classic SKU defaults to 30. You can raise it, but only to 240 seconds. That ceiling matters more than the default. No SSE stream published through Front Door survives past four minutes, so build the reconnect path and honor Last-Event-ID from the start. Otherwise, a silent disconnect looks exactly like a bug in your code.
App Service and Functions cap HTTP requests at 230 seconds. The Azure Functions hosting documentation states it plainly, and no setting overrides it. Microsoft’s own recommendation is the asynchronous pattern: return 202 Accepted and let the client poll. In other words, the platform pushes long streams back toward pattern number one.

Notice what those numbers have in common. App Service stops at 230 seconds. Front Door caps at 240. API Management accepts a higher timeout on forward-request, yet values above 240 are discouraged, because the network drops idle connections at that same boundary. All three inherit one limit from the load balancer underneath.

As a result, four minutes is the practical ceiling for a single connection in Azure. No pattern on the diagram changes that. Plan the reconnect instead.

In short, the pattern lives in your code, but the behavior lives in your platform configuration. Test through the full path, not against localhost.

Cost and scale change the answer

Persistent connections change how you scale. A stateless request occupies a replica for milliseconds. A WebSocket occupies one for hours.

Container Apps scales on concurrent requests, and a long-lived connection counts as one. Therefore, set that threshold low. Otherwise, your replicas fill with idle sockets long before CPU tells you anything is wrong.

Managed services price per unit and per connection, so check the current pricing pages before you compare. Still, the service fee is rarely the real cost. Backplane configuration, connection affinity, graceful drain during scale-in, and reconnect storms after each deployment cost far more engineering time than the invoice suggests.

Where this is the wrong answer

Every pattern has a place where it becomes a liability. Here are the ones I argue about most.

WebSockets for a dashboard that changes hourly. You pay for a connection per user to deliver one update, and you now maintain reconnect logic forever. Nobody has ever thanked an architect for putting a WebSocket on a management report.

SSE for mobile clients on unreliable networks. Reconnect works, yet you still need Last-Event-ID and server-side replay to avoid gaps. When the app spends half its day in the background, use push notifications instead.

Webhooks when ordering matters. Event Grid retries, so your receiver sees events twice and out of sequence. If order affects correctness, pull from a log or a session instead.

gRPC streaming to a browser. You need grpc-web plus a translating proxy. As a result, you added infrastructure to solve a problem that SSE already solved.

Long polling in new code. It exists for compatibility with systems you cannot change. For greenfield work, SSE is simply better.

A private socket server to avoid managed service pricing. The service fee is not what hurts. Sticky sessions, drain behavior, and reconnect storms are what hurt.

Real-time at all. If a user reads the number once per hour, batch it. Real-time is a requirement, not a compliment.

Try it yourself

The companion repository runs all six patterns from one image, against one price tick produced every second.

Because the data is identical everywhere, the six-pane test page shows exactly what differs. You watch the request counter climb under polling, the lag column drop under SSE, and the WebSocket pane accept a filter sent back up the same connection. You can also put API Management in front and reproduce the buffering failure in about ten minutes.

Deploy it with one script:

			
Bash, macOS or Linux:
./infra/deploy.sh
PowerShell on Windows:
./infra/deploy.ps1

Choose the boring one

The original advice still holds. Choose the simplest pattern that meets your latency, reliability, and scalability requirements.

I would add one line to it. Choose the simplest pattern that survives your gateway, your scale rules, and your next deployment. That constraint eliminates more options than the latency budget ever will.

Rate Limiting Algorithms on Azure: 6 Compared

Posted on September 24, 2026 by steefjan1970

Your API works perfectly, until someone hammers it with 10,000 requests in a second. Rate limiting stands between a stable system and an outage, and the usual list of rate limiting algorithms has six entries: fixed window counter, sliding window log, sliding window counter, token bucket, leaky bucket and concurrency limiter. However, few guides show what each one actually lets through, or where it should run. So I built all six in C# on Azure, load-tested them, and wrote down what I learned. The code is in github.com/steefjan1/rate-limiting-azure.

Four of the six are already in .NET

In fact, on .NET you mostly do not implement these. System.Threading.RateLimiting ships the fixed window, sliding window, token bucket and concurrency limiter; ASP.NET Core wires them in with AddRateLimiter and RequireRateLimiting. The whole fixed window:

			
options.AddPolicy("fixed-window", ctx =>
    RateLimitPartition.GetFixedWindowLimiter(PartitionKey(ctx), _ =>
        new FixedWindowRateLimiterOptions { PermitLimit = 10, Window = TimeSpan.FromSeconds(1) }));

The missing two, sliding window log and leaky bucket, are small RateLimiter subclasses in the repo. The log keeps a queue of timestamps per client. The leaky bucket, by contrast, keeps one number, the next free drain slot, and either holds the request until then or rejects it when the queue is full, making it the only limiter that adds latency on purpose.

One nuance: .NET’s sliding window is segmented, not the weighted “current plus previous window” version most explanations describe; the repo has the weighted one in Redis.

What the load test shows

Every endpoint gets the same budget: 10 requests per second per client. What matters most is the highest number of requests the backend saw inside any one-second span.

Scenario 1: a burst across the window boundary

First, a burst across a window boundary: one request opens the window, nine more arrive at 900 ms, ten more at 1100 ms.

Algorithm	Max accepted in any 1 s	Rejected of 20
fixed window	19	0
sliding window log	10	9
sliding window counter	10	10
token bucket	18	1
leaky bucket	11	0 (the last ones waited 999 ms)

The fixed window result is the textbook flaw, measured: a “10 per second” limit let 19 through in 200 ms. The log and the segmented counter, by contrast, hold at 10.

The token bucket result surprised me. A full bucket plus a couple of refilled tokens should give 12 or 13, not 18. The cause is in .NET: when the bucket is full, TokenBucketRateLimiter.ReplenishInternal returns early without moving its last-replenishment timestamp, and the PartitionedRateLimiter that hosts it refills on elapsed time. So after a quiet spell, the first refill credits all the time the bucket spent full, roughly doubling your burst allowance. The Redis token bucket in the repo, however, updates the timestamp on every call and does not do this.

Scenario 2: sustained overload

Next, sustained overload: 30 requests per second for three seconds. Fixed window and sliding log each accept 30 of 90; the segmented counter accepts 26. The token bucket accepts 39 by design, since capacity sets the burst and refill rate sets the average. Meanwhile, the leaky bucket accepts 40 with a p95 latency of 1002 ms, because it queued ten and drained them at the fixed rate.

Scenario 3: concurrency at a slow backend

Finally, 20 simultaneous calls at a backend that takes 250 ms. Without a limiter, all 20 hit it at once; with the concurrency limiter at 4, four get through and sixteen get a 429 within 2 ms. In other words, a service handling 10,000 cheap requests a minute can still fall over when 500 slow ones run at once.

The decisions that come before choosing a rate limiting algorithm

Which layer

“Combine two or three at different layers” is advice everyone gives and nobody draws. Here is mine.

On Azure, the gateway is API Management, where two of the six already live as policy XML: rate-limit-by-key counts requests per key, and limit-concurrency caps in-flight calls to your backend. Through the deployed gateway, the load test gave the same numbers for 2 to 5 ms of added latency, and APIM rejected nothing, since its policy counts only successful responses. Note, though, that rate-limit-by-key is a sliding window on the classic tiers and a token bucket on the v2 tiers: same XML, different burst behaviour.

Which unit

Ten cheap reads and ten model calls put different pressure on a system, so one threshold everywhere is the wrong shape. Instead, limit against the bottleneck: in-flight calls for a slow dependency, tokens for an AI backend, connections for a database. For example, APIM’s llm-token-limit is the token bucket with tokens as currency, and the RateLimiter API takes a permitCount, so an expensive endpoint can cost five permits while a cheap one costs one.

Where the counter lives

Every in-process limiter counts per instance. Because the sample deploys the API to Container Apps with two replicas on purpose, the same load test shows what that means: the fixed window and the sliding log each accepted 60 of 90 instead of 30, and the concurrency limiter let 8 calls reach the slow backend instead of 4. Every configured number doubled, silently, as one Log Analytics query over the API’s rejection log makes visible:

Each in-process policy rejected about 30 of 90, split across two replicas, while each Redis-backed policy rejected about 60, from one counter. APIM has the same shape: counters are per gateway node, never aggregated. Since the only way to get one counter per client across replicas is shared state, the repo runs each algorithm as one atomic Lua script in Azure Managed Redis: INCR plus PEXPIRE for the fixed window, a sorted set for the log, two weighted keys for the counter, a hash for the token bucket. Against the same two replicas, the Redis versions accepted 30, 30, 29 and 39. It costs one round trip per request, including rejects, but added no measurable latency in the same region.

Shared state, however, brings two failure modes of its own. First, clocks: the scripts take “now” from the calling replica, so drifting clocks disagree about the window; if that matters, read TIME inside the script and let Redis be the clock. Second, network: the limiter fails open when Redis is unreachable, because failing closed would be a worse outage than the one it prevents. That is a choice, and it needs an alert. Also, Azure Cache for Redis closes to new creations on 1 October 2026, so the sample uses Azure Managed Redis, clustered by default with hash tags to keep a client’s keys in one slot.

Who is being limited

Per IP punishes everyone behind a corporate NAT, and per subscription is what APIM gives. Per-user needs a validated token, so the limiter sits after authentication, and unauthenticated floods reach your identity provider. So real systems combine more than one key: IP at the edge, subscription at the gateway, user in the service. Fairness between tenants is policy, not algorithm. The sample uses an X-Client-Id header that APIM sets from the subscription ID, so the gateway and service partition on the same identity.

What the client gets

The client needs a 429 with Retry-After, or it retries at once and your limiter becomes a load generator. Every limiter computes Retry-After from its own state: the window’s remaining time, the log’s oldest entry, the leaky bucket’s next free slot.

The other half is on the caller: honour the header, add jitter, and tune retries with the limits they will hit. Microsoft.Extensions.Http.Resilience does the first two out of the box.

What you watch

Rejections and saturation are different signals. A rising 429 count per policy says a client is over budget, while a concurrency limiter pinned at its permit limit, or a leaky bucket queue that never drains, says the system is at capacity. So the sample tags every rejection with X-RateLimit-Policy and ships APIM gateway logs to Log Analytics. The repo’s docs/kql.md has the queries, starting with the one that says whether the gateway or the service produced a 429 (BackendResponseCode empty versus 429).

Where each one is the wrong answer

A fixed window is wrong when the thing you protect cannot survive 2x for a moment. A sliding log, however, is wrong at high limits: 10,000 per minute means 10,000 timestamps per client. A token bucket is wrong when the downstream needs a smooth rate rather than an average rate. A leaky bucket is wrong at an HTTP edge where clients time out before they drain; it belongs in front of a fragile dependency you own. A concurrency limiter is wrong for fairness between clients, because it says nothing about rate. And every in-process limiter is wrong the moment you have two replicas and still expect the number you configured.

What I would do

Token bucket at the gateway, keyed per client, loose enough that honest bursts pass. Concurrency limiter at the service, in front of the slow thing, sized to what it can take. For more than one replica, use a per-client counter in Redis, and return Retry-After so your own clients respect it. Two rate limiting algorithms are usually enough for a public API on a side project. Then run the load test, because two of the five let through nearly twice what their configuration says.

Cloud Perspectives

Steef-Jan Wiggers

Tag Archives: Container Apps