Email Charset Errors: Why UTF-8 Turns Into Mojibake

café usually means the original text was encoded as UTF-8, then those UTF-8 bytes were decoded as Latin-1 or Windows-1252. The fix is not “encode it again.” Find the point where bytes became characters and apply the declared character set exactly once.

In a MIME email, two headers do different jobs. Content-Transfer-Encoding tells you how to recover the original bytes. The charset parameter on Content-Type tells you how to turn those bytes into text. Decode them in that order.

Start with the damaged text, then walk backward

Here is a complete hypothetical MIME part:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

caf=C3=A9

The wire body is not the text yet. It moves through two separate operations:

caf=C3=A9
    │ quoted-printable decode
    ▼
63 61 66 C3 A9
    │ UTF-8 decode
    ▼
café

RFC 2045 defines transfer encoding as a reversible transformation back to the original octets. It even gives the useful example of Base64 bytes becoming ISO-8859-1 again after transfer decoding. The transport layer does not decide what those bytes mean.

RFC 2046 assigns that second job to the MIME charset parameter. For text/plain, the standards default is US-ASCII when the parameter is missing, and the RFC recommends that new software specify a character set explicitly.

That separation is the whole diagnosis.

Why does UTF-8 become café?

The first three letters are ASCII, so they use the same byte values in UTF-8 and Latin-1:

c   a   f   é
63  61  66  C3 A9   UTF-8 bytes

UTF-8 represents é as the two-byte sequence C3 A9. That sequence follows the multibyte grammar in RFC 3629.

A Latin-1 decoder does not recognize a two-byte UTF-8 character. It maps C3 to Ã and A9 to ©. The same bytes therefore display as café.

This Node.js fixture reproduces the boundary:

const bytes = Buffer.from('café', 'utf8')

console.log(bytes.toString('hex'))
console.log(bytes.toString('utf8'))
console.log(bytes.toString('latin1'))

The output is:

636166c3a9
café
café

This is a byte-level fixture, not a replacement for a MIME parser. Its value is that it isolates the exact handoff where the same bytes receive two meanings.

Read the shape of the damage

Different artifacts point to different failed layers:

• café, ’, or extra Â characters often mean UTF-8 bytes were interpreted with a single-byte character set.
• caf� means the decoder replaced a byte sequence it could not interpret. By the time you see the replacement character, some original information may already be gone.
• Visible text such as caf=C3=A9 means the quoted-printable transfer layer was not decoded. That is not a charset failure yet.
• A visible header value such as =?UTF-8?Q?...?= belongs to RFC 2047 header decoding, not the message-body charset path.

The last distinction matters. A subject and a body can fail independently. The guide to RFC 2047 encoded email subjects covers the header path, while quoted-printable versus Base64 owns the transfer-encoding decision.

Where should you inspect a broken message?

Keep the raw message source before opening it in an editor, exporting it, or moving it through a database. Those extra conversions can hide the first bad handoff.

1. Parse the MIME tree and select the actual text/plain or text/html leaf you are rendering.
2. Read that part’s own Content-Transfer-Encoding and Content-Type headers. Do not borrow them from a parent multipart container.
3. Decode quoted-printable or Base64 into bytes.
4. Log the resulting bytes as hexadecimal in a safe test fixture.
5. Decode those bytes with the declared charset.
6. Compare the first wrong value with the previous correct value. That boundary owns the bug.

The recursive part matters. Each MIME leaf can declare its own media type, transfer encoding, and character set. A multipart/alternative message may contain a correct UTF-8 plain-text part and a broken HTML part. The MIME structure guide shows how to locate those leaves without flattening the tree.

Can you just force UTF-8 everywhere?

For outgoing mail, my default is consistent UTF-8. Produce UTF-8 bytes, declare charset=UTF-8 on every text part, choose a suitable transfer encoding for each part, and let a maintained mail library serialize the message.

But changing only the label is not a conversion. If the body contains ISO-8859-1 byte E9 and you relabel it as UTF-8, a strict UTF-8 decoder sees an invalid sequence. In Node.js that four-byte fixture 63 61 66 E9 renders as caf�, not café.

For email you receive, “always decode as UTF-8” is unsafe. Honor a supported declared charset, reject or quarantine malformed input according to your product’s job, and record when you used a compatibility fallback. Do not silently decode, re-encode, and decode again until the text looks plausible.

There is a fair objection: many mail clients repair bad messages heuristically, so why should an application be stricter?

Because a guess that looks right for one sample can corrupt another language or destroy forensic evidence. A user-facing reader may offer a cautious fallback. An importer, archive, webhook processor, or security pipeline should preserve the raw bytes and make that fallback explicit.

What should a correct outgoing part look like?

For this example, the headers, wire representation, and decoded text agree:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

caf=C3=A9

Your test should assert both boundaries:

assert.equal(transferDecoded.toString('hex'), '636166c3a9')
assert.equal(transferDecoded.toString('utf8'), 'café')

Email verification cannot catch a broken body charset. It evaluates the address and available routing, provider, and mailbox evidence before or apart from message rendering. Keep content serialization tests beside the sending code, and keep address decisions in the verification flow.

Once the trace shows 63 61 66 C3 A9 and UTF-8 produces café, the byte-to-character boundary is correct. Until it does, keep the raw source.