Base64 with Chinese and emoji: why btoa throws and atob returns mojibake

Encode a Chinese phrase in the browser and it throws. Decode 5L2g5aW9 and you get ä½ å¥½ — characters that look vaguely like the answer but are not it. Same root cause: Base64 takes bytes and produces bytes, while a JavaScript string holds characters. Nobody decides the “characters ↔ bytes” step for you; every implementation picks one, and the default picks are wrong for non-ASCII.

Start with the right mental model: it rearranges bytes

Every 3 bytes (24 bits) are cut into four 6-bit groups, and each group indexes into a fixed table of 64 characters (A-Z, a-z, 0-9, +, /). Six bits fits exactly one printable ASCII character, which is the whole trick for carrying arbitrary bytes through a text-only channel. A trailing partial group is padded with = up to a multiple of 4: one byte short writes ==, two bytes short writes =.

So “你好” becomes 8 characters because in UTF-8 it is 6 bytes, not because Chinese is special:

你好   →  E4 BD A0  E5 A5 BD        (6 bytes)
       →  5L2g5aW9                   (8 characters, no padding)

One Han character is 3 bytes, which fills exactly one group — 3 bytes in, 4 characters out. An emoji is 4 bytes, so it costs 8 characters including two =: 🚀 is 8J+agA==.

btoa("你好") throws InvalidCharacterError on purpose

btoa and atob speak Latin-1: every code point in the string must be 0xFF or below, because one character means one byte. 「你」 is U+4F60, out of range, so the call fails. Chrome, Firefox, Safari and Node all behave the same way — it is not a broken environment, it is the API refusing to guess.

The fix is two explicit steps: string → UTF-8 bytes first, then Base64 over those bytes.

function b64encode(s) {
  const bytes = new TextEncoder().encode(s);          // string → UTF-8 bytes
  let bin = "";
  bytes.forEach((b) => { bin += String.fromCharCode(b); });
  return btoa(bin);                                    // now every char is <= 0xFF
}

function b64decode(s) {
  const bytes = Uint8Array.from(atob(s), (c) => c.charCodeAt(0));
  return new TextDecoder("utf-8", { fatal: true }).decode(bytes);  // see the next section
}

b64encode("你好");      // "5L2g5aW9"
b64decode("5L2g5aW9");  // "你好"

In Node you never touch btoa: Buffer knows its encoding, which is why the bug survives shipping — it works on the server and fails in the browser.

Buffer.from("你好", "utf8").toString("base64");        // 5L2g5aW9
Buffer.from("5L2g5aW9", "base64").toString("utf8");    // 你好

Other languages make the step visible, so it is harder to get wrong:

import base64
base64.b64encode("你好".encode("utf-8"))          # b'5L2g5aW9'
base64.b64decode("5L2g5aW9").decode("utf-8")      # '你好'

Base64.getEncoder().encodeToString("你好".getBytes(StandardCharsets.UTF_8)); // 5L2g5aW9
new String(Base64.getDecoder().decode("5L2g5aW9"), StandardCharsets.UTF_8);   // 你好

ä½ å¥½ is UTF-8 read as Latin-1

atob("5L2g5aW9") does not throw. It returns "ä½ å¥½" because it faithfully hands back 6 bytes and writes each byte as one Latin-1 character: E4 BD A0 becomes ä, ½, space; E5 A5 BD becomes å, ¥, ½. The character count is right and every character is wrong — that is the signature of a decoding-mismatch, not of corruption.

Rule of thumb: output that looks like Central European letters means the UTF-8/Latin-1 step was skipped somewhere. Output that is ??? or empty boxes usually means the source was never UTF-8 (a GBK export, for instance). The fix is not to convert again — it is to go back to whichever end produced the bytes and confirm the encoding.

Do not let a bad decode fake up characters: fatal: true

A default TextDecoder replaces un-decodable byte sequences with U+FFFD and keeps going, so feeding it an image or a zip “succeeds” and returns a wall of replacement boxes. With { fatal: true } it throws a TypeError on the first invalid sequence. An error is worth more than a screen of fake text. That is the Base64 text encoder / decoder behaviour too: real binary pasted into the decode box is reported as an error rather than turned into nonsense characters.

Length, padding, and why “mod 4 leaves 1” is impossible

The mapping between byte count and character count is fixed:

Two consequences: with padding the length is always a multiple of 4, and the cost is 33.3% inflation — a 100 KB image inlined as data:image/png;base64,… becomes 133 KB, sits inside your HTML, and loses its own cache entry. Only small icons earn that trade.

Is a length that is not a multiple of 4 undecodable? It decodes, and it is normal — plenty of systems drop =. The decoder derives the missing bits from the effective character count mod 4: remainder 2 means ==, remainder 3 means =. Remainder 1 cannot exist in any Base64 variant, so when you see it, characters were dropped in copying or the string was truncated — reporting that is more useful than guessing.

Two lenient rules matter when you paste from email: newlines and spaces are ignored (wrapping at 76 characters per line is standard for MIME), and YQ, YT==, YQ== all decode to a. But padding you do write has to be complete: the canonical form of a is YQ==, and the half-padded YQ= throws InvalidCharacterError in browsers. Omitting padding and padding wrongly are different things.

+ and / are the characters that actually break Base64

The standard table contains + and /, and both are unsafe in transit: / is a path separator, and + means a space inside a query string. Paste a token containing + into a URL parameter and the server often receives a value with a space instead — signature mismatch, and it only reproduces when the original happened to contain a +, which makes it miserable to debug.

RFC 4648 §5 defines base64url for exactly this:

standard     +  /
base64url    -  _

= is usually dropped as well — it would need percent-encoding in a query string, and as above the length tells you how many are missing. JWT’s three segments (header.payload.signature) are base64url without padding; query parameters, filenames and cookies should use it too. The Base64 tool has a URL-safe switch and reports byte count and inflation ratio; on decode it accepts both tables, so ordinary Base64 pasted in still works.

It is an encoding, not encryption

It is a public table with no key involved: 5L2g5aW9 reverses to “你好” in one step, MTIzNDU2 is 123456. Do not store passwords in it, and do not use it to “obfuscate” sensitive fields — password: MTIzNDU2 in a config file is functionally plaintext, and secret scanners find it precisely because it is one call away from readable. Use real encryption (AES-GCM and friends) when you need confidentiality; Base64’s job is to be the outer shell that lets ciphertext or binary survive a text-only transport: JSON, email, URLs, a character column.

Related questions

Why do I still get UTF-8 decode errors when I decode?

Usually a truncated copy: something between your clipboard and the decoder cut the string at a fixed column or dropped the tail. The mod 4 == 1 case is the tell — Base64 cannot legitimately have that length, so a decode error at that point means missing characters rather than a wrong encoding.

What is special about emoji?

One emoji is two JavaScript characters (a UTF-16 surrogate pair) and 4 UTF-8 bytes. Any loop that assembles bytes from charCodeAt risks splitting the pair in half; going through TextEncoder avoids the problem entirely. 🚀 → 8J+agA==.

Can I decode an image with it?

No — it only handles text. Base64 decodes to arbitrary bytes, and the header of an image is not valid UTF-8, so a fatal decoder rejects it. To read what a photo carries about how it was taken, use the image EXIF reader.

How do I decode the three JWT segments?

They are base64url, so atob needs padding restored and - _ mapped back to + /. The JWT decoder splits header, payload and signature for you and states plainly that it does not verify the signature — decoding successfully does not make a token trustworthy.

Does what I paste get uploaded?

No. The character table is hard-coded in the script, and your text is never put into a request: what you type never leaves this device.