Hacker News Reader

Comment by twic | Hacker News Reader

It's a bug with Safari because the HTML spec defines maxlength as applying to the number of UTF-16 code units [1]:

> Constraint validation: If an element has a maximum allowed value length, its dirty value flag is true, its value was last changed by a user edit (as opposed to a change made by a script), and the length of the element's API value is greater than the element's maximum allowed value length, then the element is suffering from being too long.

Where "the length of" is a link to [2]:

> A string’s length is the number of code units it contains.

And "code units" is a link to [3]:

> A string is a sequence of unsigned 16-bit integers, also known as code units.

I agree with your implied point that this is a questionable definition, though!

[1] https://html.spec.whatwg.org/multipage/form-control-infrastr...

[2] https://infra.spec.whatwg.org/#string-length

[3] https://infra.spec.whatwg.org/#code-unit

View on HN

I am not sure it’s a bug. [1] also says (emphasis added)

“User agents _may_ prevent the user from causing the element's API value to be set to a value whose length is greater than the element's maximum allowed value length.”

and we have “MAY This word, or the adjective "OPTIONAL", mean that an item is truly optional” [2]

[1] https://html.spec.whatwg.org/multipage/form-control-infrastr...

[2] https://www.ietf.org/rfc/rfc2119.txt

Apple clearly intended to implement it though, and did so incorrectly.

Clearly? Where did they say so?

I do not rule out that, if you look at the source code or git history of their code, there are comments or test cases explicitly stating they deviate from enforcing a fixed maximum length in code points if doing so breaks up a grapheme cluster.

Also: what do other browsers do when you paste, for example, an ‘é’ that is made up of two code points ‘e’ and a combining character into a field that accepts only a single further code point? Do they just add an ‘e’?

Most é (like the one in your comment) are just encoded as U+00E9 but manually creating it as a combination and testing on Chrome it does just convert back to being an e on paste.

IMHO that is then a "bug" in the HTML spec which should be "fixed" to speak of extended grapheme clusters instead, which is what users what probably expect.

Except "maxlength" is often going to be added to fields to deal with storage/database limitations somewhere on the server side and those are almost always going to be in terms of code points, so counting extended grapheme clusters makes it harder for server-side storage maximums to agree with the input.

Choosing 16-bit code points as the base keeps consistency with the naive string length counting algorithm of JS, in particular, which has always returned length in 16-bit code points.

Sure, it's slightly to the detriment of user experience in terms of how a user expects graphemes to be counted, but it avoids later storage problems.

Yeah but that assumes your database stores text in UTF-16 which seems fairly unlikely.

Consistency with JavaScript does make some sense at least.

UTF-16 seems very likely to me in database fields that cleanly support Unicode. It's the native default (for decades) in SQL Server and Oracle. Up until very recently it was the only Unicode safe character format for MySQL (when a true 8-bit UTF-8 character type was finally added after decades of hacks with broken 7-bit encodings). Postgres is the only SQL database I'm aware of that has had clean UTF-8 support for as long of a time that UTF-16 has been recommended or defaulted in every other SQL database.

Obviously there are a lot of databases out there with hacks using things like bad UTF-8 in unsafe column formats not designed for it, of course. But if you are working with a database like that, codepoint counting is possibly among your least worries.

The "length" of a string in extended grapheme clusters is not stable across Unicode versions, which seems like a recipe for confusion.

The length in code units is unambiguous and constant across versions.

Reply on news.ycombinator.com