Email Message-ID: How Threading and Duplicate Handling Work

Message-ID identifies one version of an email message, In-Reply-To normally names its immediate parent, and References carries the thread ancestry. For an inbound mailbox, help desk, CRM, or reply-by-email feature, these fields are relationship evidence. A sender-supplied ID is not a global database key, an authentication result, or proof that two payloads are the same message.

Three headers, three different jobs

RFC 5322 section 3.6.4 requires a generator to make a globally unique Message-ID. It also says a reply's In-Reply-To contains the parent's ID, while References is built from the parent's references plus the parent's ID.

The phrase “one message version” matters. Adding trace fields during transport does not necessarily create a new message identity. A substantive revision should. This is separate from MIME structure, which describes how the content is packaged inside that message. I covered that boundary in MIME Email Structure.

A five-message collision test

I ran a five-message fixture through a collision-safe thread builder. The domains below are reserved examples, and the last two messages deliberately contain failure cases.

A  Message-ID: <a@sender.example>

B  Message-ID: <b@sender.example>
   In-Reply-To: <a@sender.example>
   References: <a@sender.example>

C  Message-ID: <c@sender.example>
   In-Reply-To: <b@sender.example>
   References: <a@sender.example> <b@sender.example>

D  Message-ID: <b@sender.example>   # duplicate claim
   In-Reply-To: <a@sender.example>

E  Message-ID: missing
   In-Reply-To: <missing@sender.example>

The safe result is not “four messages because B and D share an ID.” It is five stored messages:

<a@sender.example>
├── <b@sender.example>
│   └── <c@sender.example>
└── <internal-4@receiver.invalid>  claimed <b@sender.example>

<internal-5@receiver.invalid>     unresolved parent <missing@sender.example>

All five fixture checks passed: the first valid ID stayed the lookup target, References supplied C's parent, In-Reply-To supplied D's parent, the duplicate got a new internal ID, and the unknown parent remained unresolved.

This follows the defensive cases in RFC 5256's REFERENCES threading algorithm. That algorithm normalizes IDs, assigns an internal identifier when a message has no usable ID, keeps only the first message under a duplicated ID, and creates a placeholder node when an ancestor is referenced but absent.

Receiver-side decision rules

I would keep two identifiers on every stored message:

• internal_id: receiver-generated and unique in your database.
• claimed_message_id: the normalized sender-supplied value, or null.

Then process the relationship evidence in this order:

1. Parse valid angle-bracket message IDs. Do not split on whitespace and hope for the best.
2. Normalize the ID representation before comparing it, as RFC 5256 requires.
3. Index the first valid Message-ID. Give missing, malformed, and later duplicate claims their own internal IDs.
4. If References contains usable IDs, preserve the chain and make its last usable ID the current message's parent claim.
5. Otherwise, use the first usable ID from In-Reply-To.
6. If the parent is absent, keep a placeholder or unresolved edge. Do not invent a match from the subject line.
7. Reject self-links and break cycles before persisting the graph.

A subject can help display or search, but it is weak relationship evidence. “Re: Invoice” is not a stable identifier. Two unrelated conversations can share it, and participants can change it mid-thread.

This is also why I separate header parsing from trust decisions. Start with the raw header block, unfold and parse it correctly, then evaluate the evidence. The same discipline applies when reading Received headers.

Why Message-ID cannot be your only deduplication key

It can be one signal, but it should not be the sole destructive idempotency key.

The obvious objection is that RFC 5322 requires the generator to create a globally unique ID. Why not enforce a unique database constraint and move on?

Because your receiver does not control every generator. RFC 5256 explicitly defines behavior for missing, invalid, and duplicated IDs. A safe implementation has to preserve two different messages even when they make the same claim.

For transport retries from a provider you control, prefer that provider's authenticated delivery-event ID or webhook ID, scoped to the account and endpoint. Persist the decision before acknowledging the request. My email webhook security guide shows the signature, freshness, and durable deduplication boundary.

For inbound content, combine the claimed Message-ID with receiver context and an immutable content fingerprint. A practical record might keep:

internal_id
claimed_message_id
mailbox_or_tenant_id
raw_message_sha256
received_at
parent_claim
resolved_parent_internal_id

That lets you collapse a confirmed redelivery without erasing a genuine collision. The final policy depends on your threat model and retention rules, but the internal ID must remain authoritative.

A valid DKIM signature still has limits

DKIM's h= tag lists the header fields covered by the signature. RFC 6376 section 5.4 requires From to be signed, but other fields are chosen by the signer. A Message-ID is protected by that signature only when it is included in h= and the signature verifies.

Even then, DKIM authenticates responsibility by a signing domain for the signed content. It does not turn the ID into a receiver-generated database key, prove that a person authored the message, or guarantee that two matching IDs represent one payload.

Persist the conversation edge, the authentication result, and the receiver-generated storage ID as separate fields. A later collision can then be isolated without corrupting the thread or discarding a real message.