CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root | Hacker News Reader

CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root

blog.cloudflare.com

96 pointsjgrahamc62 comments

"The Inflexibility of DNS"

This problem would not exist if clients (eg. web browsers) supported SRV record lookups. The SRV fixes a missing abstraction by mapping names that users see (eg. foo.com) to actual server names (eg. webserv42.foo.com). Without SRV record lookups, web browsers must look up the bare domain, and expect to get an address, for this to work. CNAMEs were not designed to be used for this kind of situation; it's a square peg in a round hole.

This isn't due to the inflexibility of DNS; it's due to the inflexibility of clients that ignore newer advancements in the protocol.

Of course, this does mean that web hosts are stuck.

This idea that web browsers are silly and should just start using SRV records gets a lot of traction, and I would like it if it were true. Unfortunately, the SRV specification prohibits anyone from using SRV records for protocols where the protocol’s definition does not specify the use of SRV records.

I.e. since HTTP does not specify the use of SRV records, web browsers are not allowed to use SRV lookups to locate HTTP servers.

The ones to blame are, instead, HTTP standards writers, which have not written a new HTTP standard taking advantage of the wonderful thing which SRV records are.

what about SPDY (or TLS if it's the relevant layer)?

That would cause a fragmentation where SPDY traffic might get directed to a completely different server than HTTP traffic. I mean, I know they're served from different ports already but this seem extra brittle.

the different port clearly visible since spdy URLs are https (which defaults to 443)

To be fair, the SRV spec, rfc2782, could have been written (or could be updated) to allow other, more flexible methods. Actually, that spec seems to contradict itself, saying first "...for applications where the relevant protocol specification indicates that clients should use the SRV record..." but later on the same page "The symbolic name of the desired service, as defined in Assigned Numbers [STD 2] or locally." At IANA [0] it appears that only three protocols actually use the assigned numbers infrastructure to handle this, but more could, and in the absence of new RFCs clients could adopt a convention for determining "symbolic service names" for non-SRV-aware protocols.

[0] http://www.iana.org/protocols

That’s the wrong IANA link; you want this one:

https://www.iana.org/assignments/service-names-port-numbers/...

There are not three, but thousands of protocol names registered.

You'll note http is specified there.

The list I linked to is a list of all protocols known by IANA. Whether the use of SRV records is allowed by the individual standards is not recorded by IANA; one would have to search each individual standard to find out.

HTTP, notably, does not specify the use of SRV records.

That page doesn't appear to have a thing to do with SRV.

Those are the registered service names you can make SRV records with.

If that's true, it only confirms my original point. Yet it still seems that SRV integration is a challenge for many clients.

(Oh, I see what you meant now.) The things on that page are protocols which require further allocation of numbers and/or names by IANA, and is not directly related to what protocols specifies the use of SRV records. The “three protocols” you mention are protocols which require further allocation of numbers and/or names in their use of SRV records in some way. This does not mean that only three protocols specifies the use of SRV records.

RFC2782 does not prohibit the use of SRV records with those protocols.

    In general, it is expected that SRV records will be used by clients
    for applications where the relevant protocol specification indicates
    that clients should use the SRV record. Such specification MUST
    define the symbolic name to be used in the Service field of the SRV
    record as described below. It also MUST include security
    considerations. Service SRV records SHOULD NOT be used in the absence
    of such specification.

RFC2119 defines 'SHOULD NOT' as follows:

    This phrase, or the phrase "NOT RECOMMENDED" mean that
    there may exist valid reasons in particular circumstances when the
    particular behavior is acceptable or even useful, but the full
    implications should be understood and the case carefully weighed
    before implementing any behavior described with this label.

Maybe “prohibit” was the wrong word. On the other hand, I would argue that a “SHOULD NOT” is stronger than you think: The HTTP standard (RFC 2616) says that:

“[…] the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe". This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested.

I would argue that very few people would claim that it’s OK for HTTP servers to have HEAD and GET requests be non-idempotent just because they feel like it. “SHOULD NOT” prohibitions should not be ignored just because you really want to do it.

Do you really think that's the reason? Specs should be ignored when they state things beyond their scope. The a restriction on DNS depending on if a newer HTTP spec definitely something that can and should be ignored if it helps clients.

I really think that a “SHOULD NOT” in an RFC is a sufficient reason to not do a thing, yes. I further think that specifying the usage of SRV record is wholly in scope of the RFC specifying the SRV record.

pkgng, FreeBSD's new package management tool, uses SRV records (by default) with HTTP:

  $ dig +short srv _http._tcp.pkg.freebsd.org
  20 10 80 pkg0.isc.freebsd.org.
  50 10 80 pkg0.bme.freebsd.org.
  10 10 80 pkg0.nyi.freebsd.org.
  90 10 80 pkg0.ydx.freebsd.org.

Technically, though, it's not being used by/with a web browser.

It generated enough confusion (as evidenced by the frequent "how come I can't {connect to|resolve} pkg.freebsd.org?" questions) that there's now an A RR and a page explaining it: http://pkg.freebsd.org/

I wondered how this technically works, and the answer is over here: https://support.cloudflare.com/hc/en-us/articles/200169056-C...

    CloudFlare uses CNAME Flattening to present a CNAME as an A record.

    The CloudFlare server generating the response follows the CNAME chain
    so that the response to the request from the client (for example, the
    browser) contains only non-CNAME data -- usually, an IP address. This
    effectively creates an A or an AAAA record. Without flattening,
    CloudFlare would serve the CNAME record directly to the recursor
    (the DNS resolver that translates the domain name to an IP address).

They just resolve the destination and serve it as an A record.

> I wondered how this technically works, and the answer is over here:

It was described in the article but perhaps not as clearly:

"What happens is that, if there's a CNAME at the root, rather than returning that record directly we recurse through the CNAME chain ourselves until we find an A Record. At that point, we return the IP address associated with the A Record."

> The problem stems from the fact that the DNS specification dates from 1987 ... As a result, the DNS spec enshrined that the root record -- the naked domain without any subdomain -- could not be a CNAME

This isn't due to a lack of foresight in the spec, but people's misunderstanding of what a CNAME record actually means. A CNAME says that this name refers to the exact same thing as another name, including all subnames.

You can CNAME the "bare" example.com to foo.example.org, but it would involve getting the .com nameservers serve that record instead of NS records. By having NS and SOA records for example.com, you can no longer say that the two names are equivalent. Rather than having to define a merge algorithm (eg what happens if example.com and foo.example.org both have MX records?), the RFC simply rules out additional records on the alias.

Which makes me wonder what they're doing for record types other than A/AAAA. If you have "example.com. CNAME example.net." then which record does Cloudflare provide if both example.com and example.net each have an MX record, or a TXT record, etc.? Do you get the MX record from example.com if there is one and the MX record from example.net if there isn't?

They butchered it when allowed CNAME in the root breaking RFC and other things with it, so I would not be surprised that they don't do anything, but perhaps majority only cares about A/AAAA.

This is a great example of marketing vs. technology. Cloudflare is amazingly good at marketing.

This technology is not new, revolutionary or different in any way from what a bunch of other providers do (including AWS, who they even mention in the blog post as what they are a solution for).

But yet they write it like they have solved a major problem in a new way.

Super props to the Cloudflare team! (I'm being sincere here, they did a great job with the marketing part of this product)

It is my understanding that the AWS solution only works when using Route53 with other AWS services like ELB and S3. We had a problem similar in the past and the solution was not pretty. I think they have solved the problem in a fashion that makes it easier for everyone to implement.

  > if you signed up via a hosting partner then you may have gotten a
  > CNAME that...SNIP...will now resolve directly from your domain to an
  > IP address. This makes DNS resolution a bit faster and also better
  > obscures the fact that you're using CloudFlare.

To what end are we obscuring the use of cloudflare? And how useful is the "cloudflara obscura" if the responses are being served by the cloudflare machines listed in the NS records?

  dfc@fob-xray:~$ dig +short ns nasdaqprivatemarket.com
  jill.ns.cloudflare.com.
  igor.ns.cloudflare.com.
  dfc@fob-xray:~$ dig +short ns skyprep.com
  mark.ns.cloudflare.com.
  vera.ns.cloudflare.com.
  dfc@fob-xray:~$ dig +short ns econsultancy.com
  cody.ns.cloudflare.com.
  kay.ns.cloudflare.com.
  dfc@fob-xray:~$ dig +short ns jct.com
  ben.ns.cloudflare.com.
  iris.ns.cloudflare.com.

I am not trying to be nit picky. I am genuinely curious, DNS is one of those things I find very interesting.

> "... better obscures ..."

In their defense, they didn't say it completely* obscures the use of CloudFlare.

The real question is the first question: What is the point of obscuring? I probably should have skipped the bit about the effectiveness.

As best I can tell it seems to be a very heavily veiled excuse to get their web service customers to sign up for Cloudflare hosting their DNS zones as well.

Traffic balancing and redirection via DNS responses isn't new ground--other people just don't conflate it with CNAME kludgery.

It's disappointing that prior art isn't mentioned in the article. DNSMadeEasy's ANAME record does the same. I believe other providers like Route53 also call it an ALIAS record.

It's also good to know that GeoDNS resolution won't work as expected since the location of the resolver is going to be the DNS provider and not the client.

EDIT: Actually DNSMadeEasy claims that the resolution is done locally in each of their region so the issue is somewhat mitigated.

I think it's about split between ALIAS and ANAME at the different providers.

DNSMadeEasy -> ANAME DNSimple -> ALIAS EasyDNS -> ANAME PointDNS -> ALIAS

AWS Route 53 aliases aren't ANAME/ALIAS -- they appear to only let you point a record at an AWS address, not any arbitrary IP. (Correct me if I'm wrong.)

You're probably right. I know that Route53 does support something similar but it might be AWS specific (in that it would only work with S3, CloudFront or other AWS end-points)

Disclaimer: I work on Route 53.

You can ALIAS to record sets within your own zone too, which can contain any arbitrary IP. This is useful when composing routing types (e.g. weighted plus latency). But we don't support ALIASing to record sets hosted on other DNS providers.

It is possible to ALIAS to a CloudFront distribution though, and that CloudFront distribution can in turn use any arbitrary name as its origin. It's also possible to ALIAS to an S3 bucket, which can be configured with a redirect to any name of your choice.

Remote DNS-level ALIAS is something that we're always considering, and we're interested in feedback. If we support it, it's unlikely our 100% availability SLA will apply, that there may be inter-operability issues with CDNs (though we recently announced our support for edns-client-subnet: http://aws.typepad.com/aws/2014/04/improved-cloudfront-perfo...) and incompatibilities with our future DNSSEC support.

This is good news, I recently had to migrate off of CloudFlare because of issues with CNAME records on my root[0]. I might switch back as my site is static and could benefit from being behind CF.

[0] http://joshstrange.com/why-its-a-bad-idea-to-put-a-cname-rec...

HN discussion: https://news.ycombinator.com/item?id=7293512

CF CEO commenting about CNAME flattening 28 days ago: https://news.ycombinator.com/item?id=7295215

A comment on the article makes a good point that this seems to break resolution of IPs based on geography. Perhaps CloudFlare takes this into account with their caching mechanism, but this is unclear.

It's hard to see how they could account for it perfectly. I suppose they could inject an EDNS client subnet option into the request they make upstream (with the original client's IP here), but that assumes that the target supports this option.

Resolution of IPs based on geography is already fairly broken in many cases. But it should resolve based on where the cloudflare server is, which should be ok.

That's the point. This is arguably worse than resolving based on the location of the client. Although if the server doing the caching is an edge server already located close to the client, this is less of an issue.

CloudFlare does Anycast for both DNS and HTTP so the DNS resolver will be 'close' to the client and the HTTP connection will be made automatically to the 'closest' CloudFlare PoP.

Google DNS and OpenDNS also cause some serious problems with geographic results. My go to example is www.netflix.com. I live in NY and Google DNS always points me to us-west instead of us-east.

  dfc@fob-xray:~$ host www.netflix.com # local unbound dns
  www.netflix.com is an alias for www.us-east-1.netflix.com.
  www.us-east-1.netflix.com is an alias for www.us-east-1.prodaa.netflix.com.
  www.us-east-1.prodaa.netflix.com has address 50.16.245.238
  www.us-east-1.prodaa.netflix.com has IPv6 address 2406:da00:ff00::6b14:f251

  dfc@fob-xray:~$ host www.netflix.com 8.8.8.8 # 8.8.4.4 Gives same response
  Using domain server:
  Name: 8.8.8.8
  Address: 8.8.8.8#53
  Aliases:

  www.netflix.com is an alias for www.us-west-2.netflix.com.
  www.us-west-2.netflix.com is an alias for www.us-west-2.prodaa.netflix.com.
  www.us-west-2.prodaa.netflix.com has address 54.245.96.95
  www.us-west-2.prodaa.netflix.com has IPv6 address 2620:108:700f::3270:5f36

Strange, 8.8.8.8 appears to be anycast and resolves to a east coast server for me, so I correctly get us-east-1 for netflix when I query it

FWIW, Google has proposed a solution to this problem: http://tools.ietf.org/html/draft-vandergaast-edns-client-sub...

Assuming the user is hitting their nearest CloudFlare location, this should be less of an issue.

> "The supply of IP addresses is limited"

s/IP/IPv4/

In practice today, IP basically is just IPv4. And that won't really change until IPv6 is supported by more than a comparatively small handful of networks and users.

True, but CloudFlare should be a company driving v6 forward, i.e. mentioning that the address problem only exists in v4.

Make people aware that v4 is history and present. Sure, we currently have to deal with it, but v6 and v4 should be treated equally. Esp. when talking about problems v6 solves.

They are driving ipv6 forward, they will transparently provide AAAA records for sites they host unless you opt out, and also support ipv6 only backends which they will provide ipv4 for.

Facebook has fully deployed IPv6[0], with almost all of their internal traffic now IPv6. They actually ran out of private IPv4 space to use.

IPv6 is here. If you're setting up a new service today without it you should be shot.

[0] https://www.dropbox.com/s/doazzo5ygu3idna/WorldIPv6Congress-...

This doesn't seem very... sane. It does work, but the explanation is wobbly at best. This appears to be a King Kong defense converted into a sales tactic.

In their theoretical problem, foo.com needs to be moved to somewhere else in a hurry to reduce an unpleasant load level. No problem--you simply change where foo.com points to. Normal people change the A record and call it a day. Why would you want to involve a CNAME in this? Perhaps if you needed to direct the query to some other server because it's managed by some other entity.

But wait... This requires the other entity to serve your DNS records. Not exactly a problem--even though it rather uncomfortably resembles a horizontal referral--so long as we exercise restraint, it only slows down the query a little bit. So now the lookup goes to the server your domain (foo.com) is hosted on, and needs to goes to the other server to get the foo.com.fancynametheycanmove.webprovider.com, except no... that won't really work because your DNS provider hosting foo.com would need to allow you to put CNAMEs at the root of the zone, and CNAMEs are singleton records.

The solution? Why it's to use webprovider.com to host your foo.com DNS as well! Then they can do this because their magic DNS server does all those other lookup steps for the querying client. But wait... if they're going to be hosting the foo.com zone, why don't they just change the A record in the first place? Why on /earth/ would it be useful to involve a bunch of CNAME jumps in that? For that matter, how is this even slightly usefully different than a normal global traffic manager issuing different responses for DNS based on load and server pool configuration that for the love of all that is holy I hope they already know about if they're trying to run a "cloud provider".

The reason we don't have CNAMEs in the root of zones isn't "because the standards are old" it's because CNAMEs are supposed to be a one-off which should be used sparingly because they can be used to create never-ending chains of indirection which cause serious problems for nameservers doing recursive resolution for their users. AOL used to regularly break nameservers all over the place with their casual use of endless loops of CNAME records.

Any number of companies these days don't provide you with an A record for the root ...Heroku, Amazon's ELB, etc etc etc ...the list goes on. We're solving a real problem considering how popular those services are these days. Perhaps it's not an issue for your setup, but for anyone using those names servers (among many others) it's a very real issue when you'd typically need to use a CNAME if you want to use the root vs. the antiquated WWW.

No, this completely not about the presence or lack of a 'www'.

I'm saying it's pointless to even ramble on about CNAMEs when the conditions necessary for this to /even work/ mean it would be just as easy to return an abstracted A record that points to the right/active resource in the first place... Like GTMs generally do /already/. When the query comes into the GTM for loadbalancedsite.com, the response isn't fixed--the GTM has already been keeping track of which nodes are up and/or potentially closest to the requestor and it simply hands back an A record with a reasonably short TTL without anyone having to pat themselves on the back for being 'clever'.

> We're solving a real problem considering how popular those services are these days.

In other words, it's a hack to support broken cloud providers?

Other people had this problem to. Great to see that cloudflare solves such problems.

http://translate.google.de/translate?sl=de&tl=en&js=y&prev=_...

somewhat similar to dnsmadeeasy ANAME which has been going for years?

Do clients have to know how to deal with ANAMEs? If so, this seems more backwards-compatible.

no they just exist on the server, and are mapped by their service, no client change (otherwise it would be a bit pointless...)

ANAMEs also seem to handle geolocation. http://www.dnsmadeeasy.com/services/aname-records/

  How Does This Improve CDN Services?

  When combined with the Global Traffic Director service from DNS Made Easy, ANAME
  records provide an extremely flexible and powerful solution to keep your CDN
  service running optimally. As soon as an ANAME record is defined with a
  particular GTD region (other than Default) the ANAME service ensures that
  it will only use DNS Made Easy’s resolving name servers that are in that
  particular region. For example, if you create an ANAME record with “Europe”
  as the GTD region, then only resolving name servers in Europe would be used
  to create the A records for this new record type. This ensures that your
  traffic continues to be routed regionally as planned with your worldwide
  infrastructure. Use of ANAME records are fully compatible with GTD service
  and optimize use of a CDN provider. This provides the complete power
  and flexibility that enterprise companies require from their networks today.

If you don't want to use CloudFlare as your DNS provider, you can use https://www.rootredirect.com/ to point your root domain to your ELB or other CNAME-based load balancer.

Disclosure: I own rootredirect.com.

This is a great hack, but whoever explained C pointers to the copywriter did a bad job. Maybe they confused in-process pointers with kernel virtual memory management, but the operating system doesn't reach into a process's address space and move memory blocks around and rewrite pointers. A language runtime or a memory management library can do that, but it's a little more complicated than pointer-to-a-block-that-gets-moved. It's more like using indirect or doubly indirect pointers and stuff.

Grokking pointers is hard enough as is. I weep for all the wannabe systems programmers who will get confused by this otherwise very nice announcement.

But why use CNAMEs in the first place when you don't have to?

or am I missing something?

When using a service like the AWS load balancer all you get back is an A record. You don't have a direct IP address to point to so you have to use a Cname

This is a really well written blog post. Great work cloudflare team.

Reply on news.ycombinator.com