Hacker News Reader

The SVCB/HTTPS records won't save us, because of the same reason TLS 1.3 pretends to be TLS 1.2: middleboxes everywhere have been configured to only let traffic to port 80 and 443 through. Almost everyone is running an HTTP 1.1 server so there's completely transparent fallback for those networks, but it also means those records are kind of pointless.

I never really got the point of the new records either. From what I can tell, they're just SRV records. A separate record class could've made sense with DANE, but nobody implemented it.

> middleboxes everywhere have been configured to only let traffic to port 80 and 443 through

They are not everywhere though. They are not on the greater Internet. They are not on residential networks. I have access to the full 65535 TCP ports everywhere in the world on my average Internet connection.

These middleboxes you speak of are appliances paid for and placed inside of corporate LANs. They can pay to put more modern ones, or just getting rid of them.

The point is that these middleware appliances we often use as a scapegoat exist only in well-controlled private networks. Why should we care about them?

>The point is that these middleware appliances we often use as a scapegoat exist only in well-controlled private networks. Why should we care about them?

Perhaps for the same reason we only stopped caring about IE 6 after its usage dropped below a tiny percentage.

For better or worse, making your website available to as many groups of people as possible, regardless of their browser selection or network configuration, has been a long-standing goal in the community.

If your website isn't available to folks at the offices of company X because they use appliance Y, through no fault of user Z's, well...that's not ideal, is it?

Or they could be ignored so that the users drag their IT kicking and screaming into the 21st century once an important enough site necessitates it. Corporate users usually have more direct influence on their IT than ISP customers.

https://blog.chriszacharias.com/a-conspiracy-to-kill-ie6

Exactly! That post I think perfectly encapsulates the typical line of thinking here.

> Perhaps for the same reason we only stopped caring about IE 6 after its usage dropped below a tiny percentage.

We did? Half of the sites I find posted on HN literally only work in ONE single browser.

I concur. Corporate LANs are to the internet as VMs are to servers. Trunking, VLANs, and NAT have so completely decoupled these networks from underlying infrastructure that the transport protocol itself is little more than an implementation detail. IPv4 remains king because it is well-known and cheap -- the fact that we all settled on using the same interoperable protocol for both LANs and the internet was never a foregone conclusion.

They're present often enough that the people developing the TLS spec had to invent workarounds.

Personally, I would've preferred to ignore the spec incompliant middleboxes, or even report an error about something meddling with the connection to the user when it's obvious they're the cause of any connectivity issues. That's not a very popular approach for spec designers, though, because they're afraid people will blame the new software rather than the Enterprise™ NetworkProtection© box they paid six figures for. After all, the software is what changed, not the box!

I'm with you.

Binding IP addresses into the TCP tuple is the thing that makes saddest about the could-have-been Internet.

Lack of adoption of SRV records is a close #2. So much IP space would have been saved. Load balancing and failover would have been so much easier.

(32-bit addresses is up there too...)

I feel like this is a drop in the bucket. Compared to the number of clients, the amount of HTTP servers on the internet (probably no more than a couple million if I had to guess) feels like it takes up a small fraction of the IPv4 address space.

You are right, approx 2B households alone exist.

Using SRV and increasing IP address space “by 64k” is only valuable for large single consumers such as AWS.

Most clients sit behind NAT

I am trying to parse what this means, can someone ELI5?

TLDR

For years a combination of gaps in policy, disjointed standards development, hoarding behaviour, and administrative laxity, led to substantial wastage of IPv4 address space that persists to this day, in part because multiple independent website tenants sharing an IP address was (and often still is) difficult.

Glossary

DNS: Domain Name System, how computers discover (or resolve) each other's numeric IP addresses from symbolic hierarchical names.

HTTP WG: HTTP Working Group, the standards committee(s) responsible for defining the application-layer protocols by which a web browser talks to a web server. Under the auspices of the IETF.

IETF: Internet Engineering Task Force, the standards organization for the Internet. Most famous for being the entity that publishes TCP/IP and the cherries on top.

LIR: Local Internet Registry, an entity that applies to be the holder of block-allocated IP address space. Typically ISPs and hosting companies that assign it for their use or onward customer use.

RIPE: The European peak body, a Regional Internet Registry (RIR), responsible for (amongst many other things) allocating IP space to the LIRs, ultimately under license from the global steward IANA. Sibling of ARIN, LACNIC, AFRINIC, APNIC (North & South America, Africa, Asia-Pacific respectively).

SNI: Server Name Identification. Before SNI was introduced to SSL/TLS, the only way to tell which HTTPS origin was being requested, was by server IP address.

SRV: A specific type of record in the DNS that allows lookup of a service (e.g. IMAP mail, XMPP messaging, LDAP directories) for a given domain name, to return a set of hostname(s) that actually provide that service, and the ports on which it is available. This enables multi-tenant services on shared IP addresses and thereby conserves IP space.

/19 network prefix: A large chunk of IPv4 address space, corresponding to 8192 unique IPv4 addresses. Refers to the first 19 bits of the block of (32 bit) IPv4 addresses being allocated. Back in the 90s a /19 was a common granularity of allocation from RIR to LIR and could be granted with a low bar to justification. Mathematically, 2^(32-19) = 8192.

Address record: A specific type of record in the DNS that maps a hostname to an IP address. Technically referred to as an "A record" for IPv4 addresses, or an "AAAA" record for IPv6 addresses which are literally 4x the binary length.

Allocation, Assignment, and Announcement: IP address space is allocated in blocks by RIRs to LIRs, who then assign smaller parcels of it to specific purposes such as retail hosting and internet access, and announce it at internet exchanges and to their peers for actual traffic exchange. Very often, the quantity of assignment failed to justify a large allocation, leaving space unassigned i.e. unused, and in some cases unannounced, but still hoarded, with incentives for hoarding becoming perversely stronger as the addresses ran out and the policy screws started to tighten ca.2005-2011.

Apex: in "example.com" it is the "example.com" rather than "www.example.com". For marketing purposes, almost every entity wants their zone apex to be directly reachable as a website, which means placing an address record at the apex. This excludes using the apex address record for any other service but HTTP, which I have always regarded as downright antisocial. Since alias records (a.k.a CNAME records) aren't permitted at the apex, it also makes DNS management harder when you have frequent changes to make, as in the highly dynamic world of cloud-based services, or even just want to point your website at a third party hosting service without having to edit your zones whenever their IP addresses change.

Normative: A formal declaration in one technical standard (e.g. SMTP) that it depends in whole or in part on another standard (e.g. DNS) for full specification. It is unusual for a TCP/IP-based IETF protocol standard to omit DNS as normative or otherwise specify how they use the DNS for discovery, but HTTP has always been super vague about it.

Origin: Fancy word for your actual website. Or in the presence of complications like reverse proxies/load balancers/CDNs etc, the front of your service stack from the point of view of a web browser.

Thanks for this. It seems the parent comment put maximum acronyms with zero information for people who are on the same wavelength / plane of existence.

guilty as charged

Ha, seeing the list of jargon being defined was eye opening - I understood everything without a second glance but, like you I assume, I've been exposed to this world for 25+ years as this point and its second nature.

I remember the first time I used "characters" in front of a non-technical friend instead of just the more common "letters" or literally anything else.

"octets" is a good one, especially if you have musical friends

I think "octets" is at this point historical jargon with little present value. It dates back to the day when some vendors were fighting for non-8-bit bytes. Those systems are long dead, and "byte" means "8 bits". Given that, I don't know of any context in which "octet" is an important distinction rather than an obfuscation.

Still makes a little sense as IPv4 addresses are 32 bytes and we break them down into 4 parts of 8 bits each. No one ever uses the 32 byte number, which will resolve.

IPv4 addresses are 32 bits, or 4 bytes. "octet" still isn't useful there.

On second thought I take disagreement with defining a byte as 8 bits. I would say its de facto 8 these days but not always and has always been hardware-dependent. If I give you a random page of documentation from the last 70 years of computer history and it's talking about a byte, you have no idea if that means 5 bits or 12 or anything else.... which is why I think octet is useful here as it _always_ means 8 bits and always will.

There's a reasonable distinction between "term useful to understand historical documentation" and "term useful to use in new documentation/communication". No new system or protocol or piece of documentation should ever use non-8-bit bytes. If you're reading old protocol documentation, you get used to "octet" being used in place of "byte"; that doesn't mean new documentation should cater to systems that haven't existed for a long time.

(Also, this tangent really isn't worth spending time on.)

Yet we seem to have all agreed to use octet, even for new documentation. I browsed the recent RFC submissions and had to stop looking as pretty much every one I checked that was posted this year uses octet. A few examples...

https://www.rfc-editor.org/rfc/rfc9565.txt

https://www.rfc-editor.org/rfc/rfc9598.txt

https://www.rfc-editor.org/rfc/rfc9574.txt

https://www.rfc-editor.org/rfc/rfc9582.txt

So why try to force byte into this role?

(agreed)

I meant 32 bits, of course.

so it's triplett vs octet, I see

Characters is a pretty poor term these days, since it can refer to any of bytes, code points, extended grapheme clusters, or possibly more I haven't thought of.

That was what I was referring to though - grapheme, grapheme-like unit, or symbols, letters, numbers, emoji, punctuation, etc.

people in plays

> Since alias records (a.k.a CNAME records) aren't permitted at the apex...

Technically this is a subset of the rule that CNAME ("canonical") records cannot co-exist with any other record type with the same label. e.g. it's illegal to have `foo.example.com` be both a CNAME and an A record. Since an SOA record is required at the zone apex, you can't also have a CNAME record there. :-(

not really TLDR

That it was historically "more difficult" to have more than one website per IP address particularly if it was using SSL because doing that needed amendments/extensions to DNS and other specs.

We now have it, but still need to use a hack to cname an apex e.g set Google.com to bah-bahs-tenant100.s3.amazon.com. Adhering to the spec we need to set google.com to an ip address e.g 8.8.4.4

Right now if you want to retrieve http://www.example.com, you have to ask DNS for the IP address of www.example.com. HTTP can multiplex multiple sites on a single IP address so you can make DNS answer with a "CNAME", i.e. let it give another hostname which could be from Cloudflare and will actually do the serving. However http://example.com cannot use CNAME because it is an apex record (historical limitation that's hard to lift and is worked around in many different ways by providers). And this caused many websites to occupy an IP address unnecessarily.

Likewise for HTTPS, though in that case multiplexing arrived only maybe 10-15 years ago instead of 25.

A proposed Internet "standard" suggested that instead you would do a different kind of query, not a query for the IP address but one for a "server". It's a kind of query that is very underused on the Internet but it's related to how you find printers on a local network for example. In that case you would do this kind of query (called SRV) for http.tcp. example.com and that would always be able to return another hostname, thus getting rid of the issue with apex domains.

> In that case you would do this kind of query (called SRV) for http.tcp. example.com

You’re forgetting the underscores: It’s actually “_http._tcp.example.com”. The underscores are there to avoid any possibility of collisions with host names, since host names are not allowed to contain underscores, but are generally allowed in the DNS.

> However http://example.com cannot use CNAME because it is an apex record [snip]. And this caused many websites to occupy an IP address unnecessarily.

That's not true. I have an server with a single IP and a number of Domains that are served from that single IP like http(s)://example.com http(s)://example.org http(s)://example.de

All of those delivery completely different websites from that single domain. You are mixing up two tings: DNS with HTTP(s). In DNS its right but for HTTP(s) it doesn't matter.

Of course you can have multiple apex A records that point to the same IP address. We are talking about CNAME records here. Are your sites using CNAME records or A records?

Even if you are using CNAMEs, that's only been possible recently due to hacks/workarounds, as bonzini said.

I'm imagining, but someone please correct me if I'm wrong, that this is related to the policy/standard/convention (?) that IPs have a single canonical name, so reverse and forward match. I've never really followed that strictly, though I've been aware of it and try to follow it as I can, because it makes management of DNS a little more sane.

Because of no SRV records for HTTP and no CNAME for apex records, you can't follow "one true name per IP" without giving an IP to every domain.

Actually reverse and forward match is important for SMTP these days. It is one piece in the puzzle to detect spam.

I run may own email server now for more than 20 years. Only recently I in-cooperated public RBLs to deny spam. I still don't like it to depend on external services for this, which flag spam incorrectly.

I had excluded SMTP in my thinking because SMTP has the MX records, so it doesn't run into the issue with apex records not allowing CNAMEs. If I'm mailing you at pinguts@example.com, it does not require a dedicated IP for example.com, your MX record can inform my server that SMTP needs to connect to smtp.example.net, and that needs to have matching forward/reverse, but I can have a hundred domains that all specify smtp.example.net without interfering with the apex-can't-have-CNAME rule.

I think this is what people elsewhere were referring to when they said that SRV records would be nice for HTTP.

edit: Aside: I ran my own first mail server over 30 years ago. :-)

I'm talking purely about DNS, and in fact I wrote explicitly that "HTTP can multiplex multiple sites on a single IP address", because the problem exists only at the DNS level.

A compounding issue is that CDNs (or GitLab/GitHub) want to be able to change their IP addresses, so they don't want you to use an A record.

Dns A/AAAA record = IP address

Dns SRV record = IP:port

Not sure why SRV never really caught on for most internet stuff.

(Edit: SRV records are hostnames, not IPs, so I guess it takes two lookups?)

To be precise, SRV records don't immediately return an IP address; they return canonical name(s) (i.e. FQDN of hosts) whose IP addresses can either be included in the additional part of the DNS reply or resolved separately.

In addition to canonical name & port number they also include priority & weighting values, although the usefulness of these depends on the service.

SRV might not be super visible in the application developer's lane, but swim outside and there's a lot of it about. It's foundational for SIP, for example (IP telephony). Email remains a tremendously significant internet utility and SRV is used, albeit not universally, for discovery of client endpoints (SMTP submission, IMAP, CalDAV etc). However, SRV hasn't replaced MX for SMTP between MTAs, perhaps because MX already does basically the same and was established years prior. And, well, Jabber is dead, but XMPP was another fully worked demonstration of SRV's capability.

In more local environments SRV is used for resolution/discovery in Active Directory (Microsoft) and Bonjour (Apple) - if you listen to the wire on any local network you'll often see a ton of SRV over mDNS or DNS-SD. Perhaps ironically, one of the objections that sometimes arose to using SRV for HTTP was from Active Directory sites with zone cuts at _tcp.example.com and facing additional complexity in any transition.

I never understand the reasoning behind the design decision that SRV cannot just return IP address and forcing clients to query again for A/AAAA again. That decision forces an extra RTT before connection can be established, which makes people want to avoid SRV records. Totally self-inflicted wounds.