Email Reply Parsing: Extract the New Message Without Losing Content

An email reply parser should decode the MIME message first, select one human-readable body, and only then try to separate the new reply from quoted history and signatures. The separation step is heuristic. It should return confidence and preserve the original message, not pretend every client marks replies the same way.

That is the short answer. A regular expression that splits on On ... wrote: is tempting, but it starts deleting real content as soon as the sender changes language, uses Outlook, replies from a phone, edits the quote, or happens to type a similar sentence.

I would build the parser as four boundaries. Each boundary produces an inspectable result, and none of them destroys the input.

Boundary 1: parse the message, not the string

A raw email is a tree of headers and body parts. It is not one reliable block of text. RFC 2045 defines the MIME headers that describe a part's media type and transfer encoding. RFC 2046 defines multipart relationships such as multipart/alternative.

If a message contains plain text and HTML alternatives, choose one representation for extraction. Do not concatenate both and then strip quotes. That duplicates the message before the difficult step even begins.

The nesting matters too. An attachment can contain text, and an embedded message can contain another complete email. Your reply candidate should come from the selected body branch, not every text/* part in the tree. I covered the tree relationships separately in the MIME structure guide.

This hypothetical specimen shows the boundary:

Content-Type: multipart/alternative; boundary="b1"

--b1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I can approve option B.=0A=0AOn Tue, Alex wrote:=0A> Option A or B?
--b1
Content-Type: text/html; charset=UTF-8

<p>I can approve option B.</p>
<div class="quoted-history">...</div>
--b1--

The plain and HTML parts are two renderings of the same message. A parser may prefer plain text for easier extraction or HTML for richer client-specific markers, but it should record which branch it chose.

Boundary 2: decode before looking for quotes

Transfer decoding and character decoding are separate operations. First recover the bytes described by Content-Transfer-Encoding. Then interpret those bytes using the part's declared character set. Looking for quote markers before those steps can make ordinary text appear malformed or hide the very delimiter you wanted to detect.

This is also where many reply parsers create mojibake. If you need the byte-level walkthrough, see why email charset errors happen. The operational rule here is simpler: retain the raw bytes, the declared charset, the decoded text, and any decoding warning as separate fields.

Plain text has one more wrinkle. RFC 3676 defines format=flowed, including quote depth, space-stuffing, and the conventional signature separator -- . Interpret those rules before applying line-oriented heuristics. Otherwise, a flowed paragraph can be split into false fragments, and quote depth can be misread.

Boundary 3: extract with evidence, not one delimiter

There is no universal standard that marks “the new human reply” across every email client, language, and edited conversation. MIME tells you how the message is represented. Thread headers tell you which messages are related. Neither gives you a guaranteed boundary around the newest prose.

A useful extractor combines independent signals:

• Standards-based plain-text quote depth, such as lines beginning with > after flowed-text handling.
• Recognized HTML quote containers and client-specific wrapper elements.
• Reply-header patterns, including localized variants, with surrounding date and address evidence.
• Signature delimiters and common mobile-signature shapes.
• Long runs of previously seen thread text, if your system has the parent message.

Each signal can be wrong on its own. A sender can type a greater-than sign in code. A legal footer can resemble a signature. A legitimate sentence can begin with “On Tuesday”. HTML classes can change. I would score the evidence, keep the split positions, and refuse to trim when the score is weak.

The strongest fair objection is: “Why not just split on the first reply header and move on?” Because silent deletion is worse than returning a little extra quoted text. A support agent can ignore a quote. They cannot act on a sentence your parser erased before storing it.

A parser result should expose uncertainty

The application needs more than a cleaned string. I would make the parser return a contract like this:

type ReplyParseResult = {
  rawMessageRef: string;
  selectedPart: "text/plain" | "text/html";
  decodedBody: string;
  newReply: string;
  quotedHistory?: string;
  signature?: string;
  confidence: "high" | "medium" | "low";
  signals: string[];
  warnings: string[];
  messageId?: string;
  inReplyTo?: string;
  references: string[];
};

The thread fields stay separate from body extraction. RFC 5322 defines Message-ID, In-Reply-To, and References for message identification and reply relationships. They help associate the inbound message with a conversation, but they do not prove which body lines are new. The Message-ID threading guide covers those receiver-side decisions.

For a low-confidence result, set newReply to the full selected body or route the message to review. Never convert “the heuristic did not find a boundary” into an empty reply.

Sanitize HTML after parsing, not instead of parsing

If your application renders the HTML branch, sanitization is a separate security boundary. Remove or neutralize active content, remote-resource behavior, unsafe URLs, and unsupported markup according to your renderer's threat model. Do not use HTML sanitization as a substitute for MIME parsing, and do not let it overwrite the preserved original.

A conservative system can extract from normalized plain text while retaining sanitized HTML for display. Another can use HTML structure as evidence and also produce a plain-text fallback. Both approaches can work. The important part is that the extractor never executes message content and never treats a client-specific class name as authoritative.

Test destructive cases, not just clean replies

Your fixture set should include examples where the obvious split is wrong. These are hypothetical test cases, not universal client guarantees:

Also test automatic mail before it reaches the human-reply workflow. Structured headers and report media types are stronger evidence than a subject-line guess; the auto-reply classifier shows that earlier routing step.

Choose the application policy after extraction

Reply parsing and business decisions should not be the same function. A low-risk activity feed can display the extracted text with a “show full message” control. A support ticket can keep the full body attached. A payment approval, account recovery, or legal instruction should not depend on a heuristic body split alone.

My production rule would be:

1. Preserve the raw message and a content hash.
2. Parse the MIME tree with a maintained mail library.
3. Select and decode one body representation.
4. Classify automatic messages before human-reply extraction.
5. Extract new text with multiple signals and record their offsets.
6. Return confidence, warnings, and the untrimmed decoded body.
7. Let the consuming workflow decide whether confidence is sufficient.

The success metric is not “we removed every quote.” It is “we never lose the message a person meant to send, and we can explain every boundary the parser chose.”