Events, Messages, and Delivery

Watch the Deep Dive

There's a phrase that quietly spreads through engineering teams:

It sounds clean. It sounds architectural. And the technology names reinforce it: Azure Event Hubs — surely events go here. AMQP (Advanced Message Queuing Protocol) — surely messages go here.

But events are messages. The word "message" doesn't mean "command." It means "envelope." The same trap exists everywhere — the industry labels create false categories that don't exist at the design level.

The Problem Nobody Talks About

Most systems don't fail because of bad code.

They fail because of bad assumptions about delivery.

At some point, every system runs into the same reality:

• Traffic spikes unexpectedly
• Consumers fall behind
• Failures require retries
• Ordering suddenly matters
• Someone asks, "Can we replay this?"

And that's when the cracks show.

Because the real question was never "Is this an event or a message?"

The real question was always: "What does this need to survive in production?"

Events, Commands, and Messages

A message is the envelope. What's inside determines the meaning:

Events are messages. Commands are messages. Both travel through the same infrastructure. The distinction is what's inside the envelope, not which pipe carries it. This is how production frameworks are built: NServiceBus defines it explicitly: "A message is the unit of communication. There are two types of messages: commands and events."

Commands change state. Events describe state changes. Treating them the same breaks ownership, idempotency, and debuggability.

Where things go wrong is this assumption:

That's not true.

An event can be mission-critical. A command can be low importance. The name doesn't determine the guarantees.

Even Microsoft's official documentation reinforces this confusion. Their messaging services comparison categorizes their own products like this:

Six Delivery Properties

What actually matters isn't what you call it — it's what your system needs to guarantee. Reliability lives in the transport, not in the label. Every flow can be classified by which of these six properties it requires:

Two Worlds: Streams vs Brokers

When Streams Are the Right Default

This framework is not "always use a broker." There are architectures where streams are architecturally superior:

• Event sourcing — the append-only log IS the source of truth. Log compaction retains the latest value per key indefinitely.
• CQRS read-model projection — replay rebuilds projections, ordering preserves consistency
• High-throughput ingestion — millions of events/sec for clickstream, IoT, metrics
• Log-based architectures — the log as the central nervous system (Jay Kreps' "The Log")
• Analytics and data pipelines — data lakes, ML pipelines, real-time dashboards

For organizations with mature Kafka operations, stream-first with selective broker bridging for consumers that need error handling, load leveling, or ordering is an equally valid architecture.

The Partition Math Problem

Two throughput models determine whether your consumers keep up:

Sequential Model

Batched Model

When to use which model

The partition count remains the hard parallelism ceiling in both models. Batching improves throughput per partition but doesn't remove the limit. With a broker, you can add workers freely regardless of model.

Options when consumers can't keep up:

The Bad Consumer Problem

flowchart LR
P[Producer - 50 events/sec] --> EH[Event Hub - 10 Partitions]
EH --> CG1[Consumer Group A - Payment Service - 100ms avg]
EH --> CG2[Consumer Group B - Analytics - 50ms avg]
EH --> CG3[Consumer Group C - Invoice Enrichment - 2s avg]
CG3 --> Q[Service Bus Queue - enqueue in 200ms]
Q --> W1[Worker 1]
Q --> W2[Worker 2]
Q --> W3[Worker 3]
Q --> WN[Worker N]
style P fill:#1a1000,stroke:#fbbf24,color:#f5f5fa
style EH fill:#0a1a2e,stroke:#22d3ee,color:#f5f5fa
style CG1 fill:#0d1520,stroke:#34d399,color:#f5f5fa
style CG2 fill:#0d1520,stroke:#34d399,color:#f5f5fa
style CG3 fill:#2d1010,stroke:#ef4444,color:#f5f5fa
style Q fill:#1a1040,stroke:#a78bfa,color:#f5f5fa
style WN fill:#0d1520,stroke:#34d399,color:#f5f5fa

Consumer Groups A and B keep up fine. Consumer Group C takes 2 seconds per event and falls behind at 5 events/sec max (10 partitions / 2s). The fix: enqueue to a Service Bus queue in ~200ms, then drain with unlimited competing workers. But you've just built the bridge yourself.

Where Systems Actually Break

It works... until:

The Real Tradeoffs

Neither approach is free. The question is where the complexity lives and what you get in return:

A Better Way to Think About It

Stop choosing based on "event vs message." Start choosing based on what your flow actually needs:

Quick Reference

Internal Concurrency: The Hidden Broker

There's a common response to the partition math problem: "Just add concurrency inside the consumer."

The consumer receives events, pushes them into an internal queue, and a worker pool processes them in parallel. It works. But you're now responsible for internal queuing, concurrency control, offset coordination (can't commit until all in-flight messages complete), retry paths, and ordering preservation.

Kafka Streams and Apache Flink provide framework-level solutions for internal concurrency and state management within the stream ecosystem — but these are stream processing frameworks with their own complexity, not the raw consumer model.

This is not wrong. It's a valid pattern. But it should be a conscious decision.

You Don't Eliminate Complexity — You Relocate It

Stream-based systems push complexity into consumers: internal queues, retry logic, concurrency control, offset management. Broker-based systems push complexity into platform infrastructure: queue configuration, DLQ policies, subscription management.

The architectural decision is not about avoiding complexity. It's about where that complexity should live.

The D0-D3 Taxonomy

Detailed Classification

These delivery properties map to four practical classes. The D0-D3 classification is a decision framework, not a strict industry standard. Equivalent patterns can be implemented using different technologies depending on platform and team maturity.

Decision Tree

Step 0: Does this need to be async at all? If the producer can wait and the operation completes in <500ms, consider a synchronous call. Not everything needs a message.

flowchart TD
START["🔵 New Message"] --> Q1{"Fact or Intent?"}
Q1 -->|"FACT (Event)"| GATE["📋 Catalog Gate"]
Q1 -->|"INTENT (Command)"| D3["🟣 D3: Command\nSB Queue / Topic"]
GATE --> Q2{"Ordering\nneeded?"}
Q2 -->|Yes| Q3{"Also need error\nhandling or scaling?"}
Q2 -->|No| Q4{"Error handling\nor scaling needed?"}
Q3 -->|Yes| D2["🟡 D2: Brokered FIFO\nSB Sessions"]
Q3 -->|No| KFK["🔵 Kafka\npartition-key routing"]
Q4 -->|Yes| D1["🟣 D1: Brokered Event\nSB Subscriptions"]
Q4 -->|No| Q5{"Only Replay\nneeded?"}
Q5 -->|Yes| D0["🔵 D0: Stream\nEvent Hubs / Kafka"]
Q5 -->|No| RE["🔴 Revisit\nRequirements"]
D1 --> MIR{"Also need\nReplay?"}
D2 --> MIR
MIR -->|Yes| MIRROR["🟢 Mirror\nSB → EH"]
MIR -->|No| DONE["✅ Done"]

classDef start fill:#1a1a2e,stroke:#6e6e8a,color:#f5f5fa
classDef gate fill:#1a1a2e,stroke:#6e6e8a,color:#a8a8c0
classDef question fill:#1a1040,stroke:#a78bfa,color:#f5f5fa
classDef resultPink fill:#2d1040,stroke:#f472b6,color:#f5f5fa
classDef resultAmber fill:#1a1000,stroke:#fbbf24,color:#f5f5fa
classDef resultCyan fill:#0a1a2e,stroke:#22d3ee,color:#f5f5fa
classDef resultGreen fill:#0d1520,stroke:#34d399,color:#f5f5fa
classDef resultRed fill:#2d1010,stroke:#ef4444,color:#f5f5fa
classDef done fill:#1a1a2e,stroke:#6e6e8a,color:#a8a8c0

class START start
class GATE gate
class Q1,Q2,Q3,Q4,Q5,MIR question
class D3 resultPink
class D2 resultAmber
class D1 resultPink
class D0,KFK resultCyan
class MIRROR resultGreen
class RE resultRed
class DONE done

For stream-first organizations, invert: start at D0 and bridge to brokers where error handling or scaling is needed.

The Catalog Gate

A lightweight governance checkpoint recorded before first publish. It forces teams to make explicit, defensible decisions rather than defaulting to whatever the last project used.

D0 fast path: For D0 flows, require only 4 fields: P1-P6 assessment (confirm No except P5), data owner, volume, on-call team. Full gate applies to D1-D3.

Exactly-Once note: For any flow with financial impact, exactly-once semantics must be addressed explicitly. Neither Kafka EOS nor Service Bus dedup eliminates the need for idempotent consumer logic when writing to external systems.

Re-evaluate when new consumers onboard, when incident patterns change, or annually. Consumer populations shift.

Adoption: Phase 1: Pilot with 3 teams on new flows. Phase 2: Classify top 10-20 highest-incident flows, migrate top 3-5. Phase 3: Mandate for all new flows. Requires a platform team of 2-4 engineers.

Hybrid Patterns: Combining Streams and Brokers

Most production systems need both. The question is which direction the data flows.

Pattern A: Broker-First with Stream Mirroring

Producer publishes to a broker. A platform-managed bridge mirrors to a stream for analytics and replay.

flowchart LR
    PROD["Producer"] --> SB["Service Bus
(Authoritative)"]
    SB --> C1["Consumer A
Error Handling + Scaling"]
    SB --> C2["Consumer B
FIFO Ordering"]
    SB --> BRIDGE["Platform Bridge
(Managed)"]
    BRIDGE --> EH["Event Hubs
(Replay + Analytics)"]
    EH --> AN["Analytics"]
    EH --> DL["Data Lake"]

    style SB fill:#1a1040,stroke:#a78bfa,color:#f5f5fa
    style EH fill:#0a1a2e,stroke:#22d3ee,color:#f5f5fa
    style BRIDGE fill:#0d1520,stroke:#34d399,color:#f5f5fa
						

Pattern B: Stream-First with Broker Bridging

Producer publishes to a stream. Consumers that need broker-grade delivery bridge to their own queues.

flowchart LR
    PROD["Producer"] --> EH["Event Hubs / Kafka
(Authoritative)"]
    EH --> CGA["Consumer A
Analytics - Direct"]
    EH --> CGB["Consumer B
ML Pipeline - Direct"]
    EH --> CGC["Consumer C
Needs Reliability"]
    CGC --> Q["Own SB Queue"]
    Q --> W1["Worker 1"]
    Q --> W2["Worker N"]

    style EH fill:#0a1a2e,stroke:#22d3ee,color:#f5f5fa
    style Q fill:#1a1040,stroke:#a78bfa,color:#f5f5fa
    style CGA fill:#0d1520,stroke:#34d399,color:#f5f5fa
    style CGB fill:#0d1520,stroke:#34d399,color:#f5f5fa
    style CGC fill:#2d1010,stroke:#ef4444,color:#f5f5fa
						

Tradeoffs by direction

What This Looks Like in Real Life

Anti-Patterns This Prevents

Further Reading

• Enterprise Integration Patterns — Gregor Hohpe & Bobby Woolf. The canonical text defining Event Message, Command Message, Document Message.
• Event Message Pattern (EIP) — The original pattern establishing events as a type of message.
• The Log: What Every Software Engineer Should Know — Jay Kreps. The foundational essay on log-based architectures.
• Don't Let EDA Buzzwords Fool You — Oskar Dudycz. How EDA terminology gets misused.
• Messaging Anti-Patterns in EDA — Ben Morris. Common mistakes in message design.
• Azure Event Hubs Features — Microsoft Learn. Partition model, consumer groups, throughput.
• Event Hubs Quotas and Limits — Microsoft Learn.
• Service Bus Quotas and Limits — Microsoft Learn.
• How to Choose Kafka Partitions — Confluent.
• Event-Driven Architecture — Confluent. Comprehensive intro from the Kafka company.
• Queue-Based Load Leveling Pattern — Azure Architecture Center.
• Competing Consumers Pattern — Azure Architecture Center.
• Next-Gen Event-Driven Architectures — arXiv. Benchmarking Kafka at 1.2M messages/sec.
• NServiceBus: Messages, Events, and Commands — The clearest industry definition: "A message is the unit of communication. There are two types: commands and events."
• Choose between Azure messaging services — Microsoft's comparison table that reinforces the false dichotomy this framework challenges.