🔢 Word Counter — Three Numbers for One Text, and Two Ways to Split a Word

The same text has three lengths, and all three are right

"👨‍👩‍👧‍👦 ok" — a family sign made of four people joined by invisible connectors, a space, and two letters — measures 14 in code units, 10 in code points and 4 in written units. Nothing about the text changed; the three questions are different. A code unit is what length counts, which for a JavaScript string is 16-bit storage slots, so one astral character is two of them. A code point is one entry in Unicode’s numbered list, which is what a character looks like when it is read off a code chart. A grapheme cluster is what a reader sees as one piece of writing, and that is where an accent, a flag, or a joined family collapses into a single something.

The three agree for plain ASCII and drift apart the moment text carries anything assembled from parts: combining marks, flags built from regional pairs, emoji joined with ZWJ, skin-tone modifiers. "é" written as e plus U+0301 is 2 units, 2 code points, 1 written unit. "🇨🇳" is 4 units, 2 code points, 1 written unit. A box that prints one “characters” number has picked a question; these three print the questions too, so the pick is visible.

Two word rules, and neither of them is the answer

The pattern is the usual one: a run of letters and digits, allowing an apostrophe or a hyphen inside. It agrees with what most people mean by a word in English, and it disagrees with everything else:

1.5 + 2,000 counted as four words is the pattern doing exactly what it was written to do: it stops at the dot and the comma, because a dot inside a number is not something a letter-run rule can know about. foo_bar baz runs the other way — the underscore is not in the pattern’s list of glue, so one identifier becomes two words. Chinese, Japanese and Korean are not a case of the pattern being picky; a rule about letters and apostrophes has no notion of where a Chinese word ends, so it returns nothing at all. Any word count for Chinese text is a segmentation, whether some dictionary engine does it or nobody does.

Segments are not words, which is where a first implementation goes wrong

The browser’s segmentation returns every run in the text, spaces and punctuation included, and each one carries a flag saying whether it looks like a word. don't stop comes back as three segments and two words. " \n " comes back as three segments and no word at all. Counting segments inflates the first and the second alike. The flag is what earns the difference: don't arrives as one word, because a segmenter knows an apostrophe inside a contraction is not a break, while a-b-c arrives as three, because a hyphen there is a break. The same character, two defensible answers — which is why the spans where the two rules disagree are listed rather than averaged into one number.

Which engine is answering, and where two of them differ

The word numbers come from Intl.Segmenter, compiled into whichever JavaScript engine is reading the text with you, so they are as much a fact about that engine as about the text. Measured on the same fixtures, Chrome 150 and Node 22.20 agree on every row above and differ on one more: in https://a.b/c Chrome splits the host at the dots (https, a, b, c) while the Node build keeps a.b whole. Asking the segmenter in English, in Chinese, or with no locale at all changed none of these results — the segmentation of CJK comes from a dictionary compiled in either way, not from the locale named.

What counts as one Han character, and what a range list forgets

The Han count asks a script property rather than a range list: is this code point one of the characters of the Han script. The difference is measurable and one-sided. 𠀀𠀀x — two ideographs from Extension B plus a Latin letter — contains 2 of them, and a range list that stops at U+FFFF finds 0, because Extension B begins at U+20000 and every character up there occupies two code units. 〇⿰。（ contains exactly 1: the ring 〇 is Han, while the ideographic description character, the full stop and the full-width bracket are not, however much they look the part in a Chinese sentence. Hiragana, katakana and Hangul score 0, because they are different scripts writing different languages, and a count of 汉字 that folds them in is counting something else.

An engine too old to answer the property question falls back to the range list. The number that comes out is then shown with a warning whenever the text reaches past U+FFFF at all, because such a count may be missing ideographs and does not say so quietly.

Lines, and the whitespace that is not a break

Line counting splits at CRLF, CR or LF, so a is one line and a\n is two — the second one empty, which is what a line count means when the text ends in a break — while an empty box reports 0 rather than the 1 that splitting an empty string hands back.

The box settles that question before the count is asked: a text box rewrites every CR and every CRLF into a single LF, so a\r\nb\rc arrives as five characters and three lines, measured in Chrome 150 whether the text is set into it or typed into it. Windows text pasted in is therefore counted on the same breaks as Unix text.

U+2028 and U+2029 are not breaks to this count: two figures with a paragraph separator between them stay one line, however an editor renders them. That is a definition, not an oversight — the three the rule names are the breaks that keyboards and file formats really produce, and text carrying the other two is rare enough that a count disagreeing with an editor about it is the smaller surprise.

Whether anything was typed at all is judged by the whitespace class, and that class is wider than the space bar: it covers the no-break space, the ideographic space U+3000 that Chinese input methods type, the thin spaces, U+2028 and U+2029, and the byte-order mark. A box holding nothing but those is answered as if it were empty — no sentences about the numbers, while the numbers themselves still show, because a box of invisible characters does have a length. Every count here includes all whitespace; nothing is trimmed away before measuring, so the code-unit number is the number of slots the text occupies and not the number of keys that made it.

What leaves the device

The counting is done to the text in the box, in the tab, by the rules written out above; the only outside thing in the arithmetic is the browser’s own segmentation engine, and it is handed text that never leaves. Nothing you type or paste here is sent anywhere or stored.

Questions

Q: Which word count is the real one? Neither. The pattern’s number is what a letter-run rule gives, and it is the number most writing tools print; the browser’s number is a segmentation, and it is the only one of the two with any notion of where a Chinese or Japanese word ends. Pick by what the number is for: a limit stated in words by someone who types English, or a rough sense of how much text there is in a language the pattern cannot read at all.

Q: My Chinese text shows 0 words by the pattern. Is that broken? It is the pattern working. It matches runs of Latin letters and digits, and there are none to match. The other column is the one that counts Chinese, and it does it by segmentation, which is a dictionary decision rather than a character class.

Q: Why do the two word numbers part company on a document full of figures? Because 1.5 and 2,000 are one word each to a segmenter and three and two pieces to a letter-run rule. A page of numbers, IDs and thousands separators is where the two counts drift furthest apart.

Q: How big can the text get before something gives out? The walk covers the first 262,144 code units and says so when it stops there; the difference table lists the first 24 spans and gives the real total above them. Both are about staying responsive while typing — a box filled to that ceiling counted back in about half a second in Chrome 150, and the text pasted in the box is never modified.

Q: Is the written-units number the one to quote in a contract? Quote the one the other party reads the same way. Written units is what a reader counts in front of them; code units is what a buffer holds; code points is what a Unicode table numbers. Three sentences that look identical can have three different lengths, which is the whole reason the three are printed side by side.