Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost"
twitter.com
twitter.com
And this is despite that fact that the actual "font color" is #14171A with excellent contrast: https://contrast-ratio.com/#%2314171A-on-white
But these bizarre characters don't respond the text color and thus utterly unpredictable in their legibility.
> these bizarre characters
At least on Twitter.com, they're not characters, they're SVG images.
At first, I was thinking that I could see the difference quite clearly between the two colors and I had no clue what you were talking about. Then I hit the "swap colors" button and happened to notice that there was a bunch of text on the left side that I hadn't even seen...
Hawaiian uses ' as a proper letter, granted.
None of this detracts from the greater point, being that we need a way for all writing systems to somehow squeeze down into the subset of ASCII supported by legacy protocols.
But then again, when you invent something, you decide how it works, and the world could just re-invent a more international version of DNS but chose not to.
Also, ASCII is still probably the best charset to use, if you have to choose one; it can represent most possible sounds in some way and symbols represent single sounds (as opposed to, for example, chinese characters). It's very widely used (as opposed to, for example, the greek alphabet).
So yes, limiting DNS names to 26 latin characters and 10 arabic digits was probably the best option at the time.
The Latin system is the one you're using here. It's very popular. Most humans in the integrated tribes are somewhat familiar with it‡. It has twenty six "letters" and then twenty six more "capital letters" which look different but mean almost the same thing for some reason, and then a bunch more symbols that aren't quite letters although some (apostrophe, ampersand) have a better claim than others. But other popular systems include Han, which has a shitload of logograms, and Cyrillic and Greek which have different letters than Latin and different rules about how letters work.
Anyway, the people who invented DNS only or primarily used the Latin system and they weren't much into capital letters. So, their system doesn't treat capital letters as different and only has one set of twenty six Latin letters, ten digits, a dash and an underscore for some reason.
But, lots of people who do NOT have Latin as the primary or only writing system found this annoying to work with. They wanted to use their writing system with DNS especially once the Web came along and popularized the Internet.
Punycode is a way to use some reserved nonsense-looking Latin text in any DNS label to mean that actually this DNS name should be displayed as some Unicode text. Unicode encodes all popular human writing systems (and lots of unpopular ones) fairly well, so this sorts the problem out. Specifically Punycode reserves Latin names that start xn-- for this purpose. By only caring about display this avoids changing anything in the technical underpinnings. Only user-facing code needed to change, every other layer is unaltered.
The rules about IDNs say that a registry (such as .com) should have rules to ensure the names in that registry are unambigous and meaningful. But in practice the purpose of the .com registry in particular is to make as much money as possible regardless of the consequences. So you can register any name you please and they won't stop you even if it's deliberately confusing.
‡ None of the extant unintegrated tribes have writing. This probably doesn't mean anything important.
I'm having trouble confirming this as the double-hyphen is now very much allowed, thanks to IDN.
Viewed in that light, restricting DNS names to ASCII as a way to reduce bugs, security issues, etc., makes a lot of sense.
People thought you would follow links from directory pages linked from your "home page", hence the house symbol still seen in browsers. Yahoo! is a leftover of an attempt at a directory.
It's kind of like XML. It was never meant to be seen by human eyes, except in a few debugging scenarios. Unfortunately, that intention was ignored, and now we have a usability disaster. (At least in the case of XML, it can just die.)
The worst part of it is that it doesn't have a stable definition for numbers, making it impossible to guarantee you're getting the same value back if you encode and then decode a number. Reliably preserving the data you serialise using it should be a primary feature for an encoding format. JSON can't even do that.
My opinion is that a serialisation format that explicitly makes something as fundamental as the representation of numbers unspecified is not useful as a data representation format.
You were meant to surf the web ^^.
> so this sorts the problem out.
Assuming you can't read Arabic, how on earth would you even recognise that address, let alone remember how to type it.
I always assumed the non-latin-alphabet people know their way around this because they already used the internet before unicode-domains.
Much easier to just allow everything in the technical standards and let domain registries set reasonable standards that make sense for that tld. Sometimes that works well (.de for example has a list of 93 allowed unicode characters [1] which covers everything a German might plausibly use from German or neighboring country's alphabets), but some registries just don't seem to care much (e.g. .com).
1: https://www.denic.de/en/know-how/idn-domains/idn-character-l...
But that's not a very good reason to continue it (besides which, there are new people coming to the Internet every day).
So Punycode wasn't specifically designed to work with emoji, it just happens to work with emoji because emoji are part of Unicode.
address=/:xn--*:/0.0.0.0
does anyone know if something like this is possible with unbound?1. Use a browser extension that throws a warning on all unicode domains (maybe even with unicode highlighting). Drawback: Needs to be done per-device.
2. Let your pihole MitM all https traffic with a certificate you do NOT trust (maybe create one per domain, so you can add it to the trusted list); if the connection is over http, upgrade it to https (if the server doesn't speak https, proxy it). Drawback: It's much more complicated, and if your bank happens to be called e.g. "Bank of Zürich" you still need to take a look at the IDN to determine if you're on the right website (or add an exception).
You probably don't want to remove the dots on that one.
Similarly for a power company, http://xn--rsted-uua.dk, just a redirect, but they do use it on adverts and my electricity bill.
Some that don't redirect: http://xn--mgk--jra.dk/ https://www.xn---strm-uuae.dk/ https://xn--magnusbrth-85a.se/
(HN has converted the displayed URLs to Punycode, presumably as a quick security measure without reference to the reasonable characters for each TLD.)
To tell you the truth IDN domains feel like a failure, a gimmick. Their biggest market probably was meant to be countries that don't use the Latin alphabet, and they've failed spectacularly.
If you use Firefox, for your own security, set network.IDN_show_punycode to true.
They seem to be used in Russia a little more, especially with the .рф TLD[1], although many are still just redirects.
I have no idea what the text means, but these sites look reasonable: https://www.xn--80aicstx0byb.xn--p1ai/ http://xn--j1abth1c.xn--p1ai/ https://xn--d1ai6ai.xn--p1ai/
[1] https://en.wikipedia.org/wiki/.%D1%80%D1%84
(HN seems to have translated these to Punycode, presumably a quick security measure without respect for non-ASCII languages.)
so I hacked something together that uses the linux kernel NFQUEUE: https://github.com/DyslexicAtheist/nfq
this way I have guarantee that these domains will never be resolved (which is what I want :))
Browsers kept insisting on showing this as xn--b38haa.crankybill.com, so I went with "grr" instead.
Do browsers always show the Punycode encoding, or do they show the encoded glyphs only in some scenarios? I can't find examples of Punycode in the wild used by normal websites.
There already is an emoji for apple (the fruit, not the company). Oh the horrors. I should start an emoji NIC!
Just a reminder that this is backwards, not forwards: this was my first experience of the Internet, through AOL, and I imagine I'm not the only one.