Hacker News Reader

Comment by landofredwater | Hacker News Reader

What would the mechanism behind case-insesitive string compare be? How would you program out all the edge cases?

By implementing the Unicode’s case folding specification in detail, as described in section 5.18 of the Unicode standard.

Or, more likely, by using a library like ICU.

it is host specific.

Unicode does not depend on a host. You probably meant something else, and the expression came out wrong.

Care to explain in more detail?

As Unicode standard describes (e.g. the same 5.18 section mentioned above) case mapping depends on locale, so lowercasing the same string may have different results on different hosts, and so also the truthfulness of lowercase(x)==lowercase(y) is not universal and depends on the host locale.

See the standard https://www.unicode.org/versions/Unicode11.0.0/ch05.pdf for the most commonly used example of Turkish i, but there are others.

Indeed, a fundamental to the problem is that most unicode text doesn't actually carry the relevant locale information... (Of course, one probably wouldn't want to rely on sender-specified locales for email adresses when deciding address equality -- that would open one up to all sorts of potential weird scenarios, i.e. a nightmare for security).

If you're setting up proper case folding, part of your job is not leaving locale up to the host.

Presumably the case-insensitive version is also doing unicode normalization as well, which is what a byte-level comparison of tolower versions would miss

A primer (taken straight from GP's first link):

> the full case foldings are superior: for example, they allow "MASSE" and "Maße" to match.

This sounds like a fun vulnerability to find in a password reset flow

Yeah but isn't email address in ascii? I still have no idea why it would be different.

Another falsehood to add to the list.

> isn't email address in ascii?

Yes in the basic SMTP RFC, but there are extensions that allow non-ascii local-parts.

Reply on news.ycombinator.com