Hacker News Reader

Comment by jph | Hacker News Reader

An email falsehood surprised me recently: I thought a case-insensitive email address can be compared by using pseudocode `lower(x)`. But that's false.

An email system that guarantees case-insensitive email addresses can still fail during comparisons of lowercase-to-lowercase, due to international encodings, locales, I18N, L10N, etc.

Pseudocode:

    string-compare-case-insensitive(x, y) => true // Right way

    string-compare(lower(x), lower(y)) => false // Wrong way

It turns out this issue is called a "case folding non-deterministic" error, and is a broader issue with strings in general.

For more about "case folding" with Unicode: http://www.unicode.org/Public/UNIDATA/CaseFolding.txt

For more about "non-deterministic" comparisons: https://www.postgresql.org/docs/current/collation.html#COLLA...

View on HN

I thought a case-insensitive email address can be compared by using pseudocode lower(x)

You shouldn't be comparing the mailbox part of email addresses at all other than as literal bytestrings: you cannot know what equivalence rules the mailserver for that domain uses.

The domain part can be equivalence-tested using the normal rules for domains though, including case insensitivity, IDN translation and punycode resolution.

> you cannot know what equivalence rules the mailserver for that domain uses.

You're right in general. In my specific post, I do know the equivalence rules, because I'm administering the mail system and working with its source code, and the documentation guarantees/requires that internally its email addresses are treated entirely as case-insensitive.

What I saw was source code comments about not using `lower(x)` nor Postgres module `citext`, and instead using Unicode case folding ICU and Postgres non-deterministic collations. In the end, what surprised me wasn't about email servers in general, it was about human languages with case folding.

> You shouldn't be comparing the mailbox part of email addresses at all other than as literal bytestrings

It's hard to know what to do in practice, but this seems to be wrong according to https://www.rfc-editor.org/rfc/rfc6532#section-3.1: "normalization form NFC SHOULD be used".

This also depends a lot on why you are comparing those addresses. If you want for example to make sure that you don't easily allow the same person to register multiple accounts (say, to take advantage of a free trial period), then they are both wrong, since X@gmail.com, X.@gmail.com, X..@gmail.com etc. are all the same account and cost nothing to make.

However, if you just want to make sure this is the same user that signed in earlier, you get to chose what rules you want - it's their problem to some extent to remember what account name they gave you.

Just a nitpick: Gmail doesn't allow consecutive periods, so X..@gmail.com doesn't work. You can do a.bc, a.b.c, ab.c etc. instead.

Oops, you're right. Did they also support user+somestring@gmail.com to be equal to user@gmail.com as well?

Yeah, they support +something. Sadly lot of websites don't accept it as a valid email address because of the +.

I wonder if that's because they know people use + to tag for later spam blocking, or is it just a lazy regex user who needs to read this article?

IME it's the latter, but not because they are lazy, but because email validation is hard. The lazy regex was in the late 90's early 00's when any email that didn't end in .com, .edu or .net failed!

The problem is insufficient laziness. Just check that the email contains an `@` sign, and validate later by corresponding with the address.

    if "@" in email:
      send_confirmation(email)

Which is truly terrible. I mean, here's a valid email that was used in the early 90s:

    sdchema!mhtsa!australia!chris@Nosc

Our website blocked it inadvertently, we were using ASP.NET Identity which has internal email validation which by default does not accept it. Just flipping a flag in the configuration is enough to allow it, but until someone reported the problem I wasn’t even aware it was blocked.

Found a mailing list ("marketing") service that allowed you to sign up with a +, but any 'unsubscribe' was a link to a URL with your email in it, with the + sign in it, and... the unsubscribe page could magically never find your address to unsubscribe you. I submitted this as a bug report to the service, and was brushed aside. At that time, I also had a couple friends who worked there, and tried to run it up the flagpole there, and I was told about half the folks there didn't understand the problem, and the other half didn't really care, because the + trick was seen as very niche. So I started reporting all the newsletters I got as 'spam' - well after I'd tried to unsubscribe.

You could sign them up using many + emails

Did you try to replace the + in the unsubscribe URL with %2B (the urlencoding of a +)? Because interpreting the URL on the server will almost certainly mean urldecoding the arguments which will turn + into a space.

Doesn’t the browser do this for you?

Nope, cause + used to mean space. So plus symbol has sometimes to be left alone, some times replaced - due to still supporting legacy stuff and the newer RFC that makes + go away.

FWIR, yes. And it didn't work. They had some weird parsing going on to deal with some weird legacy stuff from years earlier, and... it made life more complicated than it needed to be.

I ran in to the exact same thing.

My solution was subscribing "legal+i_bet_you_will_fix_it_now@service.com" to the same list.

I just wrote a post[1] about using the user+alias@example.com instead of user@example.com. Basically treat your "regular" email address as a spam target, and use user+alias for all wanted communications. The +alias can be a random string, making it harder for spammers to guess.

This works in gmail for both sending and receiving, although sending as you+alias@gmail.com requires messing around in the settings[2] and having to remember to select a non-default From: address in your outgoing mail.

[1]: https://alexshroyer.com/posts/2022-09-01-Default-Honeypot.ht... [2]: https://webapps.stackexchange.com/questions/3598/is-there-an...

If this gained widespread adoption, then spam would just +string and you'd have to allow-list.

Allow-list isn't the worst, but this isn't a long-term solution in and of itself

True, but the nice thing is it that allow-lists can be handled transparently by your email client.

Never saw this +string before? Mark as spam.

Is the +string missing? Spam.

Recognize this +string from your "Sent" folder? Keep.

Signing up for something? Generate a pseudorandom +string and add it to the allow-list.

Actually RFC 5322 says that you can't have two consecutive periods.

What RFC 5322 says you can do and what Google will allow you to do with an @gmail.com user part are two different things.

Except in this case, where they're the same thing.

Since Gmail allows this.

Not all providers are Gmail.

Which makes the problem even worse. Are you supposed to have comparison rules per provider then? Only for those that are large enough to bother in your particular use case?

It just means you cannot use email validation as a way to limit one account per person. A person could just have another email on another provider anyway.

It's part of the reason why you see many places ask for your phone number, not that I agree with that, and even this method has many flaws.

My favorite issue with phone number-based identity verification is that it doesn't account for multiple different countries well at all. In some regions, getting more than one 'legit' phone number (i.e. not on any of the VoIP blocklists) per person is nearly impossible, while in other regions, even in the Western world, it's very much possible to buy a lot of pay-as-you-go plans, keep the numbers from expiring, and use them to retain perhaps dozens of accounts at no to minimal cost, making it yet another measure that affects legitimate users disproportionately compared to it affecting illegitimate users.

I really wish there was something that was easy to get as a company but hard to get as an individual. As a B2B SaaS, I want to do the organization-level equivalent of KYC — "Know Your Corporate Buyer"? — but there's no such thing.

Instead, all anyone offers in this vein are identifiers that are really annoying to get and that nobody already has as a matter-of-course of registering a company; such that many real companies can't (or won't bother to) pass them. DUNS numbers are a common choice. Our company is five years old and doesn't have a DUNS number. It takes two weeks to get one. So how could I expect our customers to bother getting one just to try our product?

"can you set this value in a TXT field on your DNS" or another domain-level challenge-authentication mechanism would be one. Sure, anyone can buy a domain but in theory that is what Extended Verification is supposed to represent, so if you get a valid handshake from an EV domain then in theory you're talking with someone representing the organization that the EV certificate was issued for.

Of course in principle you can get an EV cert for any company as well, but now we're talking about determining whether a company is sufficiently well-known to accept, which is not something that can fully be solved deterministically since that's a human judgement, there will always be some grey areas. But, there are certainly a lot of companies that could probably be "automatically verified" in some fashion (ford motor company? reasonably well-known, and this is their domain...) given some sort of authoritative domain -> stock mapping, and that's not impossible to do if you trust the EV scheme.

Anyway, not perfect, but challenges based raises the bar a lot from "register domain with godaddy and sign up for let's encrypt" to "first you have to get an EV certificate..."

Of course, since such a mechanism is not widely used... not exactly going to find tons of official support for using it like that. But the mechanisms are there!

Isn't a company itself a thing that is easy to get as an individual, in certain jurisdictions?

Yeah, but not usually easy enough that you can make hundreds/thousands of them quickly for a single purpose and then throw them away.

Also, as a slight tweak, I'd hope that this authentication scheme would only admit legal companies that are at least a week old. People who do these sorts of bulk-registration attacks tend not to be patient people, willing to wait around for their credentials to gain reputation before using them. They create them and then try to use them right away. Whereas no real company would be signing up for most B2B services in its first week of legal existence. (Google Workspace? Sure. Accounting software? Probably not.)

Ultimately this depends on what the attack is.

If it's targeted to X company for a specific reason, they'll wait however long they need. If it's generic and X company is just caught in the crossfire, then maybe.

>Whereas no real company would be signing up for most B2B services in its first week of legal existence. (Google Workspace? Sure. Accounting software? Probably not.)

When I've registered companies in the past, it's when I have sales. Before then, it's just an idea to see if we get users. But the company with the service can determine that.

Corporate Number/Tax ID?

You can, however, use email parsing + heuristics as a way to detect people who are trying to register accounts that look like they ever could be part of a set of "related" addresses — and just reject even the first member in that set. (I.e. just reject registrations with a + in the username part in the first place.)

(You don't have to be petty about it; no need to send them to a "you're an attacker" page or anything. Just redline the form-field, explain the problem clearly, and let them modify the address until it's less duplicable.)

Yes, this doesn't stop people from manually registering multiple times, since, as you say, people can have multiple addresses; or even email service from multiple providers. But it does stop low-effort automated bulk-registration attacks. And some services — those with any sort of free-tier especially — get a lot of those!

That + sign as a label to the same inbox is a Gmail thing.

It's a 100% valid character to use. Doesn't have to mean it's a label in another hostname. Goes back to what others have said. You either end up with a ton of rules, or go with the fact that email isn't the best solution to identify unique users.

https://en.wikipedia.org/wiki/Email_address#Local-part

> The local-part of the email address may be unquoted or may be enclosed in quotation marks.

>If unquoted, it may use any of these ASCII characters:

> - uppercase and lowercase Latin letters A to Z and a to z

> - digits 0 to 9

> - printable characters !#$%&'*+-/=?^_`{|}~

> - dot ., provided that it is not the first or last character and provided also that it does not appear consecutively (e.g., John..Doe@example.com is not allowed).[5]

> If quoted, it may contain Space, Horizontal Tab (HT), any ASCII graphic except Backslash and Quote and a quoted-pair consisting of a Backslash followed by HT, Space or any ASCII graphic; it may also be split between lines anywhere that HT or Space appears. In contrast to unquoted local-parts, the addresses ".John.Doe"@example.com, "John.Doe."@example.com and "John..Doe"@example.com are allowed.

(it keeps going on the wikipedia page)

I know that it's allowed; but — given that we're a B2B company serving highly-technically-literate customers — I don't think I want the business of anyone who thinks it'd be a good idea to use it even when it is allowed, given how it'd affect their own deliverability to imperfectly-implemented MTAs.

(It's a similar "you're really relying on perfect competence from the whole rest of the ecosystem to get you out of this" feeling as e.g. putting a space in your user name — and thus your home directory path — in Linux. Sure, the core GNU/BSD/etc tooling has been tested for that use-case — but are you really going to trust random tools and shell-scripts to handle argument tokenization perfectly? Or are you just going to ditch the space to be safe?)

>serving highly-technically-literate customers

It would probably be the other way.

The more technical literate a user is, the more they'll understand and figure out how things work, the nuances, and use those things.

You know that core tooling that works? It's because technical literate users, tried it, found the issue, and fixed it.

Same way with me using é on my name. If something breaks on a website, email, etc, I create an issue and starts emailing. My name doesn't have an e, it has an é.

Yes, if you want to know if two addresses refer to the same account, you need to keep up with the actual rules for each provider that you decide to care about.

But, if you're using the email as a username just for convenience, you get to decide what rules you have for username comparison. You can absolutely decide that A@gmail.com and a@gmail.com are different accounts for your service, even though emails you send to these accounts will likely reach the same inbox. You can also decide that they are the same account, but that haşim@gmail.com, HAŞIM@gmail.com and HAŞİM@gmail.com are all different accounts. It's entirely up to you to know what is important for your users.

In this case, a@googlemail.com is also the same as a@gmail.com

you are supposed to not use email as a token of user uniqueness, because it's not.

the fundamental problem here is assuming that "emailA == emailB" could ever conceivably be used as a proxy of the equality "userA == userB". Unicode tarpits aside (that's a legitimate problem where even if you did everything exactly right, libraries or api calls etc might not) the idea of "do these mailboxes belong to the same user" (in the sense of stripping '+' suffixes from gmail boxes etc) cannot be answered period. You cannot affirmatively confirm this nor can you disprove the assertion either, without some additional information about the behavior of the receiving mail transport agent.

even if you understand the behavior of the receiving MTA, you cannot prove that they don't have an email account somewhere else, so that is not sufficient either.

email is not a key for user uniqueness, period. It's a channel for contacting a user. You don't mess with it, and you simply send an email and see if the session user can authenticate it. If you want identity verification that happens at another layer, not email.

"transforms" like lowercasing or mailbox-suffix-stripping are only window dressing patching around this fundamental issue. There is no way to affirmatively go from "email address" to "user identity" without more information from another layer. It's a communication channel, not an identity provider. You can (reasonably) use it as a medium to talk to the user while you verify them against something else (although like SMS it can of course be hijacked unless your IDP protocol accounts for this possibility) but "email verification" is not and can never be an IDP in itself. Gmail addresses are free.

I don't think I know a single person, tech-savvy or not, that only has one email address.

> (in the sense of stripping '+' suffixes from gmail boxes etc)

I have a few accounts. Those + suffixes drive a bunch of rules on them to get me to take action. They forward to my primary email address if it's important.

Some of them get scraped and mailed in a weekly summary. Some of them send text messages alerting me.

When they mess with them, I'll probably just not see it unless I have to login for some other reason to that account.

Yes - other providers have their own strange ideas of what it means for two emails to be equal.

My point is that there is no standard that you can use to decide if two email addresses refer to the same inbox.

Each provider has their own rules for what emails they consider identical, and you need to either invent your own rules, or learn what theirs are, depending on why you are doing this.

Using Unicode aware string comparisons doesn't solve the issue any more than byte-wise comparison or ascii case-insensitive comparison.

But you can't just say "if hostname == gmail.com", since it should also be done for any domain that uses Google Workspace for email.

Of course. That's the problem.

You can't reliably say that a.b@ and ab@ are the same user except in a very small population (gmail.com) because you don't know how other domains will handle it.

Email is not a unique identifier which is how they're wanting to use it.

GMail users are anything but a small population.

Not users wise, but domain wise. Just one domain vs all the domains out there.

Badly worded on my part.

The whole thing is a mess. I remember trying to make a Postgres email address column and wanted to do make sure it can do comparisons either way, then found this stackoverflow post that shattered my expectations of a clean well understood problem: https://dba.stackexchange.com/questions/68266/what-is-the-be...

Yes you're right, same kind of issue. Thank you for the link with the detail.

I think the part before the @ is actually case sensitive per RFC, but most mail server will treat it case insensitive. But I am not sure I am reading the RFC correctly, citation:

Verbs and argument values (e.g., "TO:" or "to:" in the RCPT command and extension name keywords) are not case sensitive, with the sole exception in this specification of a mailbox local-part.

rfc 822 local-parts (stuff to the left of @) are weirdly permissive.

you can include quoted spaces, carriage returns and newlines (!)

Quoting CR and LF is no longer allowed in RFC 2822.

Correction: it's still allowed, only considered obsolete. :-/

Which is to say, nobody of note is still allowing it; you can allow it if you like, for anyone who is still doing it; but if you don't, and someone sends you a message, and it bounces, then, well, they already know why.

Yes, there are very few rules on the standard that could create two binary-different local parts that resolve to the same value (mostly involving parenthesis). But the mail server can add as many rules it wants, including the very common setting of making completely different names resolve to the same mailbox.

Case insensitivity was a huge mistake in computing really. Most languages don't have cases and its very non trivial to convert between cases. Should have treated every char as completely unique.

Sure. At the user signup side, block emails that are too close in ways like case, but as a sender you should always treat them as unique emails.

tolower() works mostly well in ASCII and languages which don't have weird rules (=works mostly well in english).

The context is e-mail addresses, not English.

I think that's GP's point: tolower() looks like it works well to English speakers but it's subtly wrong and will fail unexpectedly for people with other locales.

Those rules are only weird to you, but perfectly reasonable to others, such as Turkish or German speakers…well, readers :-)

Internalization is hard. I think its too much to expect software written for a specific market to handle all languages in the world.

Fx in danish we have 3 letters (æøå) that is not common in the latin alphabet. I cant go to germany or turkey and expect people to be able to write out those letters when doing input in a local system.

Fun thing I've run into in a Germany-based but increasingly international company: German always spells out umlauts and eszetts when going to lower ASCII for email addresses ("Schäßler" -> "Schaessler"), but Hungarian does not. Not sure how Turkish ö and ü get fully lower-ASCII-ized there, but in Germany, they get spelled out "oe" and "ue" as if they were German ö and ü. This isn't as much of a corner case as one might think - there are a lot of people with Turkish names in Germany.

I’m always amused by these kinds of nonsensical usage for Turkish in Germany (ü->ue) but the thing that really trips people up is that Turkish has two letters that look like i, one with the tittle and one without — in both cases.

Germany relatively adopted an uppercase ß (and got it into Unicode) to try to help with case roundtripping but I’ve never seen it in the wild. And let’s not get into obsolete German Fraktur ligatures like tz or ch which also had no upper case equivalents.

> Germany relatively adopted an uppercase ß (and got it into Unicode)

The parenthetical part is true. It was an uphill battle, but not because of the consortium, but because of what the tropes wiki would describe as executive meddling.

The adoption is not recent, but about 110 years old. You have the wrong idea because of sloppy journalism.

> I’ve never seen it in the wild

I see it all the time. Maybe you are undercounting. Pay attention to non-standard letterforms on hand-written signs, and you also have to include print media where someone substituted lower-case ß in absence of a glyph in a font. This is a typographic mistake, but the intent is clear.

> to try to help with case roundtripping

I don't think that's right. It's used in certain contexts like all caps.

> as a sender you should always treat them as unique emails

This is already how it works. Senders are not supposed to assume that local-parts are case-insensitive. (Some buggy implementations ignore this requirement and upper-case everything, but the serious implementations don’t).

That's a surefire way to increase customer support load, as users have mix case for their emails all the time. They might sign up on a phone, or login on a phone. They might just be hamfisted. Sure, it might be their problem, but they'll make it yours.

And people forgetting caps lock down for their passwords is so common that many UIs actually show a little caps lock icon in there to warn you

*accidentally enable caps lock

Some places even support the same password with inverted caps. So say, if your password was "passWORD", then "passWORD" and "PASSword" would work. If the first hash fails, they'll invert the case, re-hash, then check again.

So if my email is HeLLo@example.com because I want to be cute people will have to try 6 times before they finally get the right email address? Imagine telling that to someone in person. This kind "weird" casing isn't that rare and doesn't require cute usernames: "DonaldDuck@example.com", "FreeBSD@example.com", "DrMcCoy@example.com", etc.

Languages that don't have case is not an issue; the situations where a lowercase <-> uppercase mapping is not simple are actually not that many. It's not trivial, but not all that complex either. The most annoying part is Turkish, Azeri, and Lithuanian where the rules differ a bit but the used language is often unknown. For the purpose of matching things ("is this email address known in our system?") it's actually not that hard, since you can just treat several characters as identical (displaying text correctly to users is harder, but that's not important here).

I see this attitude in various situations, often under "falsehoods programmers believe" articles, which goes something like "it's hard in a few rare cases, therefore we should not do it at all for the >99% cases where it's simple and unproblematic".

> So if my email is HeLLo@example.com because I want to be cute people will have to try 6 times before they finally get the right email address? Imagine telling that to someone in person.

Yeah. That's fine.

If my email is LLLLLLLLLLLL@example.com because I want to be cute I have to tell people to type exactly the right number of L's. Do you think they should just be able to type a lot of L's and as long as it's somewhere near it counts as the same email address?

In a world where email addresses are always case sensitive, everyone will use lowercase (like they pretty much always already do anyway), and it'll be fine.

"LLL@" doesn't map to "LLLLLLLL@" in any logical way. "lll@" does; that's just a silly argument.

It also maps to 111@ and III@, depending in font

By maps to, they mean it counts as the same email address. 111@ and lll@ do not do that. The font has no impact on the email spec. However it can add extra confusion.

I doubt anyone is crazy enough to implement the email spec for comparing emails, to be fair. I would honestly be surprised if any publicly available mail agent or server supports that craziness.

The only reason "LLL" and "lll" mean the same thing are because currently email addresses are (sometimes) case insensitive.

In a world where email addresses were "obviously" case sensitive, "LLL" mapping to "lll" would be just as crazy as "LLLLLLLLLL" mapping to "LLLLLLLLLLL".

They just seem similar, to humans. But they're different strings.

"everyone will use lowercase (like they pretty much always already do anyway)"

This in itself sounds is a falsehood!

It is fine to have multiple email addresses connected to a single inbox. Email providers already do normalization like this that is not baked into the spec. Gmail for example treats johndoe@gmail.com and john.doe@gmail.com the same.

And that’s why I regularly receive other people’s mail in my gmail inbox, and why i have stopped using gmail for anything important (it’s right to assume that gmail is also sending my emails to other people).

Google’s gmail people aren’t really as smart as they think they are.

That's something I don't understand. I've always given my email as john.doe@gmail.com, and I sometimes receive emails - addressed to Another John Doe - sent to johndoe@gmail.com.

That Another John Doe never, ever had access to johndoe@gmail.com, they just gave a wrong address. That's not gmail's fault.

Or someone assumed (or just tried) that other email address.

I signed up for my university's email forwarding for alumni early on and got my first name as my email. For quite a while, I would get emails, including fairly sensitive ones, sent to me by not yet very email savvy people just assuming you could send an email to someone's first name and it would get to them.

Nah, it happens with mangled names that no bot would ever try to stuff, too. E.g. I own derefr@gmail; but I sometimes receive email from people trying to reach a man named "Derek" — who almost certainly owns the address derek.fr@gmail, but probably typoed it once as dere.fr@, and now his browser autocompletes that into registration forms for him.

> Or someone assumed (or just tried) that other email address.

I received 2-3 personal emails, but most of them are automated invoices.

That Another John Doe never, ever had access to johndoe@gmail.com, they just gave a wrong address. That's not gmail's fault.

This. My wife and I have two flavors of this.

Her address is firstmlast@gmail.com. There people frequently forget the m initial and somebody else owns firstlast@gmail.com She's since started using dots first.m.last to mitigate the error.

My address is firstlast@gmail, where first and last are not globally common, but are fairly common in Scotland. Once a year or so, I receive email for somebody else that shares my name. I don' know his real email, but I've been "invited" on his family vacations 3-4 times now. Infrequent enough that I just respond "thanks for the invite, but I think you'll be disappointed when I arrive and not the Alistair you were expecting."

I have a friend that has firstmlast@ and she's friends with firstlast@ because of how common the issue is.

It's actually not a super common first and last, so firstlast@ knows when to who to forward.

This doesn't really make any sense. It's not just gmail that does this, dots are almost always ignored before the @.

Nobody else can register an email that is the same as yours but without a dot. So the only way you receive someone else's email is if they give the wrong address.

> It's not just gmail that does this, dots are almost always ignored before the @.

That's not my experience. Which non-gmail email software ignores dots before the @?

Thinking about this, I guess the sending MTA doesn't care about dots; it goes RCPT TO: <address.with.dots@example.com>. The receiving MTA then has to validate that address; it does that using some account database that isn't typically part of the MTA - it could be a unix account (no dots!), a database table, or an LDAP user. Finally it passes the mail off to a delivery agent, which hopefully relies on the same account database.

So the elision of dots appears to be a feature of certain account databases. So which account databases elide dots?

MTAs can be configured to additional transforms before looking up the account. For example, postfix's virtual table [0] can be used for this and on my server it does elide dots in the local part (along with everything else).

[0] https://www.postfix.org/ADDRESS_REWRITING_README.html#virtua...

Dots are never ignored before the @, and also aren’t ignored after it, for that matter.

I guess this is another falsehood people believe about emails.

> Nobody else can register an email that is the same as yours but without a dot.

It used to be possible, then google decided to stop allowing that (guess why?)

And by the way, that’s an arbitrary decision.

I have run mail servers and it’s just and cam tell you… it’s an arbitrary decision.

that cannot be true. I think you meant john+doe

edit: wow. goodbye gmail

"if your email is johnsmith@gmail.com, you own all dotted versions of your address:

john.smith@gmail.com jo.hn.sm.ith@gmail.com j.o.h.n.s.m.i.t.h@gmail.com"

>johnsmith@gmail.com and j.o.h.n.s.m.i.t.h@gmail.com are the same address and go to one inbox

https://support.google.com/mail/answer/7436150

No, it's true. Can place a dot anywhere and it arrives in the same inbox.

It is true and it's not just gmail, dots before the @ are ignored.

It may be that some other specific email servers implement similar behaviour to gmail, but that is not true as a general rule.

Oh but that is true.. :)

https://support.google.com/mail/answer/7436150

Well, maybe Google should check their own implementation, because interesting things happen when accounts for both version exist.

I have an account with the dot, that was made in age, when Gmail was invite-only. Few years ago, someone created an account without the dot. Yes, I'm receiving their mail, and have no way to contact them, because everything I send out comes back to me.

Most likely no account without the dot actually exists, it's just someone having written their email wrong at some places.

Then that guy puts the wrong address consistently in places, where you want to receive the mail, from applications to education courses to paypal.

Yes, people do that.

Not only do people do this, it's actually extremely common, ask anyone with commonname@gmail.com. Someone here said their original email address has become unusable because it gets thousands of messages a day that are intended for other people. "That guy" might actually be multiple people all making the same mistake with your email address.

My coworker told me there's a complete stranger who, every time she emails her son, accidentally sends the email to him first. This has been going on for years, she makes the same mistake every time, she doesn't learn her son's actual email address, and she doesn't learn to press "reply."

I'm 100% sure the "non-dot-version" doesn't exist as a separate account.

AOL used to allow users to email other users without using the @aol.com extension. Back in those days I (prior to any capacity to negotiate a sensible ISP, for the record) had an email that matched a common subject line that was inundated by people who typed their subject in the To line and then wrote an email.

> Case insensitivity was a huge mistake in computing really

original ASCII only had uppercase. When lowercase got added, gradually with newer systems and software, without case insensitivity you would have had massive incompatibilities which probably would have hampered or even arrested the introduction of lowercase in new systems

and all over again, when microcomputers first came out, they came out with uppercase only to cut complexity and cost on simple systems.

This was the original mistake. Encoding the font into the letter representation.

Possibly, but this dates back to .. maybe the 7th century and Carolingian Minuscule?

Capital letters aren't a matter of font. There's a difference between the river phoenix, a magical bird which lives by the river River, and River Phoenix, the actor. It isn't a presentation-layer difference, it's an encoding-layer difference.

From wikipedia: "A capitonym is a word that changes its meaning (and sometimes pronunciation) when it is capitalized."

All bets are off at the beginning of a sentence.

I don’t see how adding lowercase to ASCII could have resulted in massive backwards compatibility issues considering that uppercase-only ASCII existed for only a few weeks in 1963. Surely there was not widespread adoption of ASCII in the spring of 1963.

I'm not old enough to be an expert, just old enough to have used the leftovers (new computers were too expensive!): many many devices were uppercase only, card punches, ASR-33 teletype, the Telex system, lineprinters, "glass" teletypes, FORTRAN, COBOL, and now that I think of it, Morse code/telegraph had always been. There was a ton of infrastructure that was uppercase only. You may be right that it wasn't ASCII's fault. Perhaps the first version of ASCII made sure to encompass what had been, and then saner heads said "let's allow for future progress".

i'm not going to explore the entire history, but just looked this up. TL;DR example, the addition of lowercase characters represented a jump from 6 bits to 7 bits at the hardware level:

"A six-bit character code is a character encoding designed for use on computers with word lengths a multiple of 6. Six bits can only encode 64 distinct characters, so these codes generally include only the upper-case letters, the numerals, some punctuation characters, and sometimes control characters. The 7-track magnetic tape format was developed to store data in such codes, along with an additional parity bit."

https://en.wikipedia.org/wiki/Six-bit_character_code

Do you have any data to back that most languages dont have cases?

Only latin and Cyrillic scripts (or more broadly, alphabetic systems) have cases. Abjads, abugidas and logo/syllabic systems don't have cases.

Most alphabets don’t have case (Devanagari, Arabic, etc) and many don’t have letters at all (Chinese, Japanese, et al).

Case is only used in a few alphabets, mainly Latin-, Cyrillic-, and Greek-derived ones.

Are they a majority of languages if counted? I guess it also matters if you count the number of languages or if you count the number of people writing them.

Let’s just use Greek and its descendants (Latin, Greek and cyrillic alphabets) and Brahmic-derived writing (we said “alphabet” when I was a kid but now ppl say “ Abugida“. There are about 200 languages spoken in Europe, all of which use these alphabets. India has over a hundred “major” languages and about 1600 others, most of which use Bramic writing alphabets (the major exception, Urdu, uses a form of Arabic writing). So a big imbalance!

Oh, you want speakers? merely counting people who read Hanzi + Arabic-Alphabet readers + the Indian subcontinent gets you more half the world‘s population. And there are hundreds, maybe over a thousand writing systems.

> in Europe, all of which use these alphabets

Latin is much more wide-spread than that! You neglect whole of America and Australasia and half of Africa.

https://upload.wikimedia.org/wikipedia/commons/9/9d/Writing_...

> So a big imbalance!

When we sum up realistically, then world-wide the amount of users of writing systems with case/"cameral" are about equally balanced with those without.

It's just blown my mind that case might be a thing non-English speakers would need to learn to be able to read and write English. (Same for non-English, but that doesn't blow my mind in the same way.)

Yeah, we always say that the English alphabet has 26 letters, but there are actually 52 unique symbols you have to learn to read, or 104 if you also have to read/write cursive. Some of these symbols are very similar (if you learn 'o' you will definitely recognize 'O', and likely the cursive variants as well), while others are quite different ('g', 'G', and the cursive upper case G might as well be different letters altogether; the lower-case cursive does resemble 'g').

With joined-up letters (“cursive” in the USA I guess) different languages have different letterforms, and sometimes multiple systems.

For that matter typesetting rules vary by language as well — not just the obvious hyphenation rules busnspacimg as well. Just pick up a book in, say, French or Russian and you can tell at a glance (without even looking at the letters) that it’s not in English.

Right, 104 symbols would be the minimum if you do need to read/write cursive.

However, I don't agree with your point about typeset text. You're right that the styles differ, but if you have learned one style, and know the language of the text, you will not need any significant amount of time to read a different style of typesetting.

Russian of course normally uses the Cyrillic alphabet, not the Latin one, so obviously you do have to learn a whole new set of symbols to understand it even if you can read Latin symbols. And of course French uses slightly more letters/letter forms than English, with the sedile and four accents (egu, grave, circonflex, and very rare treme).

Lots of accents when using Cyrillic to write non-Russian text.

I didn’t mean the typesetting differences made reading a different language in any way hard, merely pointing out that there are lots of different aspects to text in different languages even when the alphabets are basically the same.

And there are a fair number of (inconsistent) rules for casing. Proper nouns vs. common nouns. Camel case (or other non-standard capitalizations). Title case. "Standard" body copy.

I would probably count stuff like abugidas and kana as letters in that context.

I think you have correctly identified an implausible claim!

Of course, most languages aren't written at all ... or at least don't have a traditional written form that is sufficiently well established for someone to say that the "language" has case rather than a particular (proposed) way of writing it.

However, I rather suspect that the majority of languages in which books are published use some variant of the Latin alphabet and do, therefore, have case. (The only language I've heard of that uses the Latin alphabet without case is Lojban!)

On the other hand, if you weight languages by the number of (native) speakers, since about three quarters of the world's population lives in China, India, Pakistan, Bangladesh, Japan or Korea, probably it's true that most people don't use case in their main language.

The correct answer is "who cares" though. Languages which use cased alphabets.. use cased alphabets, you don't get to argue with it.

You also don't get to argue with the fusional position changes in Arabic, or the ligatures in Devanagari, or the places within a square the featural particles of Hangul must be printed in.

These things aren't negotiable.

You are correct that its not negotiable when supporting that language, but it is negotiable what languages and writing sets a given application support.

> Case insensitivity was a huge mistake in computing really

Ah, youth. There was little choice! Sometimes you had only six bits for a character; sometimes your bytes could be from 1-36 bits wide, depending on what you wanted for your program, so you might have systems that mixed six-bit (only upper case) and early ascii (two cases) and so for matching you had to be case insensitive.

It’s easy to look back and say “those people were so stupid” but they weren’t.

Computer have to adapt to people and not the other way around.

E-Mail wouldn't have been widely accepted with case sensitive addresses.

People expect that MyName@mail.com reaches the same person like myname@mail.com or Myname@Mail.COM just like letters reach their recipient irrespective of the upper and lower case of the recipient's name.

Computers are there to make the lives of the users easier, not the programmers.

I'm not sure this is true. In my experience a lot of people actually do think that it's case sensitive. Many times I've heard someone describe the capitals while verbally telling someone their addresss.

However it being insensitive has probably helped a lot of times where people make mistakes in explaining or copying those capitals.

Regarding email it doesn't hurt to think they are case sensitive but opposite would be a massive problem

As a sender, you should always treat email as case sensitive. As an email host/receiver, you can and probably should chose to be insensitive. But never assume any other host works like that.

Similar to how gmail ignores . in emails but other hosts do not.

> In my experience a lot of people actually do think that it's case sensitive.

I don't think they really do. They may think case matters somehow, and so may be careful to reproduce the exact case that they used before, but I don't think many people would expect JohnDoe@gmail.com and johnDoe@gmail.com to be too different email accounts.

In the general population, how many people do you think understand that username (including an email address) probably isn't case sensitive but that password almost certainly is?

People accepted telephone that used just numbers, they would accept case-sensitive e-mail as well. Everyone would just use small caps.

I think anyone who deals with end users would disagree. It seems impossible to get users to abide to a specific casing. Things would break all the time.

Then again, if we could start from scratch we'd probably just have a single global phonetic language without case and with a limited number of total chars.

Honestly non-phonetic glyphs are probably an easier lift.

Fun fact: the reason we pronounce "ph" like "f" is because the Greek letter was originally pronounced like p-h, at the time Romans began stealing words, but then the Greeks started pronouncing it like "f" and the Romans followed suit, but kept the old Latinization of "ph", because they'd already carved it into stone.

English spelling is largely phonetic... but it captures the phonetic spelling across dozens or hundreds of shifts in the spoken language. Unless you can stop people from changing how they speak, any phonetic spelling reboot is either going to suffer from the same problem, or words will constantly change how they are spelled to keep up with the spoken word.

China has the proper idea for a writing system this entire time.

if you view meme culture as a trend towards increased symbol density in linguistic communication due to their ability to convey emotions, overtones, implications, and other nuance ("shaka, when the walls fell") then the increased symbol space of chinese/japanese/korean characters looks interesting.

conversely it's certainly been an obvious disadvantage (posed a lot of problems and imposed a lot of awkward workarounds) for mechanical/electronic communication - now you have to enter the characters too, and you have to express that larger number of characters efficiently. In practice, a lot of electronic communication is just simplified to ASCII because that's the set that works universally. Someone used the example of ess-tset being transliterated as "ss" in german, dunno if chinese uses anything similar, but it wouldn't surprise me, obviously Japanese has romaji too.

but at a human-interface level, fundamentally there is a limit to how many symbols people can absorb. Even with latin characters, people at best will sight-read whole words to increase symbol rate, but, the natural evolution is to use 1 character to represent 1 symbol/word, that's the highest possible rate at which humans can absorb symbols for a written system. And in turn you could in principle absorb a "word of symbols", which is a sentence, similar to how western readers can sight-read a word of our 1-character glyphs.

by "increasing the dimensionality" of the symbol, you increase the effective symbol rate, similar to how memes use subtext/etc to convey more nuance than a pure text can by itself.

Kids learn phonetic writing much quicker than glyph writing. It's much more intuitive for a child.

It also helps to limit the fragmentation of pronunciation of words.

> string-compare-case-insensitive(x, y) => true // Right way

what is inside that function?

    func string-compare-case-insensitive(x, y) {
      return lower(x) == lower(y) ? lower(x) == lower(y) : lower(x) == lower(y);
    }

More seriously, if you want to learn about this, one place is to look at OpenJDK's sources. E.g. here:

https://github.com/openjdk/jdk/blob/master/src/java.base/sha...

which calls: https://github.com/openjdk/jdk/blob/17283cfe4c697e2118f19992... and https://github.com/openjdk/jdk/blob/17283cfe4c697e2118f19992...

What's inside that function is Unicode case folding and non-deterministic collation, such as ICU, or its equivalent. See https://icu.unicode.org

What would the mechanism behind case-insesitive string compare be? How would you program out all the edge cases?

Presumably the case-insensitive version is also doing unicode normalization as well, which is what a byte-level comparison of tolower versions would miss

By implementing the Unicode’s case folding specification in detail, as described in section 5.18 of the Unicode standard.

Or, more likely, by using a library like ICU.

it is host specific.

Unicode does not depend on a host. You probably meant something else, and the expression came out wrong.

Care to explain in more detail?

As Unicode standard describes (e.g. the same 5.18 section mentioned above) case mapping depends on locale, so lowercasing the same string may have different results on different hosts, and so also the truthfulness of lowercase(x)==lowercase(y) is not universal and depends on the host locale.

See the standard https://www.unicode.org/versions/Unicode11.0.0/ch05.pdf for the most commonly used example of Turkish i, but there are others.

Indeed, a fundamental to the problem is that most unicode text doesn't actually carry the relevant locale information... (Of course, one probably wouldn't want to rely on sender-specified locales for email adresses when deciding address equality -- that would open one up to all sorts of potential weird scenarios, i.e. a nightmare for security).

If you're setting up proper case folding, part of your job is not leaving locale up to the host.

A primer (taken straight from GP's first link):

> the full case foldings are superior: for example, they allow "MASSE" and "Maße" to match.

This sounds like a fun vulnerability to find in a password reset flow

Yeah but isn't email address in ascii? I still have no idea why it would be different.

Another falsehood to add to the list.

> isn't email address in ascii?

Yes in the basic SMTP RFC, but there are extensions that allow non-ascii local-parts.

Reply on news.ycombinator.com