Hacker News Reader

Comment by jonhohle | Hacker News Reader

It seems odd to suggest the bug is with Safari. Normal humans (and even most developers!) don’t care that the byte length of an emoji in a particular encoding that may or may not be under their control defines the maximum “characters” in a text box (character used to define a logical collection of code points each of which may fit into one or multiple bytes).

View on HN

It's a bug with Safari because the HTML spec defines maxlength as applying to the number of UTF-16 code units [1]:

> Constraint validation: If an element has a maximum allowed value length, its dirty value flag is true, its value was last changed by a user edit (as opposed to a change made by a script), and the length of the element's API value is greater than the element's maximum allowed value length, then the element is suffering from being too long.

Where "the length of" is a link to [2]:

> A string’s length is the number of code units it contains.

And "code units" is a link to [3]:

> A string is a sequence of unsigned 16-bit integers, also known as code units.

I agree with your implied point that this is a questionable definition, though!

[1] https://html.spec.whatwg.org/multipage/form-control-infrastr...

[2] https://infra.spec.whatwg.org/#string-length

[3] https://infra.spec.whatwg.org/#code-unit

I am not sure it’s a bug. [1] also says (emphasis added)

“User agents _may_ prevent the user from causing the element's API value to be set to a value whose length is greater than the element's maximum allowed value length.”

and we have “MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional” [2]

[1] https://html.spec.whatwg.org/multipage/form-control-infrastr...

[2] https://www.ietf.org/rfc/rfc2119.txt

Apple clearly intended to implement it though, and did so incorrectly.

Clearly? Where did they say so?

I do not rule out that, if you look at the source code or git history of their code, there are comments or test cases explicitly stating they deviate from enforcing a fixed maximum length in code points if doing so breaks up a grapheme cluster.

Also: what do other browsers do when you paste, for example, an ‘é’ that is made up of two code points ‘e’ and a combining character into a field that accepts only a single further code point? Do they just add an ‘e’?

Most é (like the one in your comment) are just encoded as U+00E9 but manually creating it as a combination and testing on Chrome it does just convert back to being an e on paste.

IMHO that is then a "bug" in the HTML spec which should be "fixed" to speak of extended grapheme clusters instead, which is what users what probably expect.

Except "maxlength" is often going to be added to fields to deal with storage/database limitations somewhere on the server side and those are almost always going to be in terms of code points, so counting extended grapheme clusters makes it harder for server-side storage maximums to agree with the input.

Choosing 16-bit code points as the base keeps consistency with the naive string length counting algorithm of JS, in particular, which has always returned length in 16-bit code points.

Sure, it's slightly to the detriment of user experience in terms of how a user expects graphemes to be counted, but it avoids later storage problems.

Yeah but that assumes your database stores text in UTF-16 which seems fairly unlikely.

Consistency with JavaScript does make some sense at least.

UTF-16 seems very likely to me in database fields that cleanly support Unicode. It's the native default (for decades) in SQL Server and Oracle. Up until very recently it was the only Unicode safe character format for MySQL (when a true 8-bit UTF-8 character type was finally added after decades of hacks with broken 7-bit encodings). Postgres is the only SQL database I'm aware of that has had clean UTF-8 support for as long of a time that UTF-16 has been recommended or defaulted in every other SQL database.

Obviously there are a lot of databases out there with hacks using things like bad UTF-8 in unsafe column formats not designed for it, of course. But if you are working with a database like that, codepoint counting is possibly among your least worries.

The "length" of a string in extended grapheme clusters is not stable across Unicode versions, which seems like a recipe for confusion.

The length in code units is unambiguous and constant across versions.

Yea the authors conclusion is flawed. If I enter an emoji and a different one appears I'm just going to assume your website is broken. Safari is in the right here.

It's not that clear-cut. If you enter that 11-code-unit emoji into a field that has maxLength=10, and the browser lets you do so, but the backend system that receives the data only stores 10 code units, you're worse off -- because you probably won't realise your data got corrupted -- than if the browser had prevented you entering/submitting it.

Reply on news.ycombinator.com