Hacker News Reader

Comment by eastdakota | Hacker News Reader

Hi: I'm the co-founder and CEO of CloudFlare. This can be a problem with some mail servers (especially Exchange, in our experience). Good news is we have a solution. We have a feature called CNAME Flattening which dynamically turns a CNAME at the root into an IP address. This solves the issue. We've beta tested this for the last few months and it works great. We haven't exposed in our UI yet but if you write to our support team we'll enable it. We're offering it to all our customers, even as the free tier. We plan to roll it out in our UI soon. Functionally, it's very similar to services that have an "Alias" record without requiring users to learn a new, non-standard DNS record type.

Matthew Prince (@eastdakota) Co-founder & CEO, CloudFlare

View on HN

The problem doesn't have much to do with particular mail servers, but rather with the order in which queries may be made. Let's say the following two records are allowed to co-exist:

    example.com 60 IN MX mail.example.com
    example.com 60 IN CNAME foo.example.org

and that the MX record takes precedence over the CNAME when queried. Imagine a client, C, behind resolver R, and it makes the following queries;

    1. C looks up qname=example.com qtype=MX, R returns the MX record
    2. C Looks up qname=example.com qtype=A, R follows the CNAME
    3. C looks up qname=example.com qtype=MX, R follows the CNAME

The resolver will first resolve the MX record, and cache that correctly, passing the result back to the client. The second query though will result in a CNAME, which will likely invalidate the cached MX record, and the resolver will then follow the CNAME and chase that. The third query may now also follow the CNAME, even though there may not be an MX record at the CNAME target (foo.example.org).

Any non-MX query behind resolver R effectively poisons R's cache, and the MX becomes unreachable. If R gets no queries other than MX queries, or if the MX and CNAME target answers are cacheable for identical periods of time, then it can sometimes work - but it's a tenuous and non-deterministic stroke of luck. If there's even one lookup behind R - from any of its clients for a non-MX, things break again. Some mail servers do perform an A lookup before an MX lookup (which is valid RFC822) and so things never work for those mail servers, as they always poison the cache, but that correlation isn't really the cause of the problem. The configuration is unreliable even in the best of circumstances.

Full disclosure: I work on Route 53.

Excellent explanation, thank you. :)

I don't understand what you mean by cache poisoning. If a resolver was queried for a single record type, that doesn't mean that's the only record that resolver will return... That would break all kinds of things. Usually DNS servers return referrals to other record types if they don't find the one they're looking for, but in your example the correct record type always exists, so it should always be returned and cached properly. The mail server may decide to error out if it finds a CNAME where it expected only A and MX records.

edit Oh, I see, this is a DNS spec issue, not an MTA or resolver issue. According to DNS specification clarification RFC2128, if a CNAME exists, no other record should exist with that name - so in theory a DNS server or resolver could simply drop all other records if it found a CNAME. (That seems like very bad behavior, though!)

  10.1. CNAME resource records

   The DNS CNAME ("canonical name") record exists to provide the
   canonical name associated with an alias name.  There may be only one
   such canonical name for any one alias.  That name should generally be
   a name that exists elsewhere in the DNS, though there are some rare
   applications for aliases with the accompanying canonical name
   undefined in the DNS.  An alias name (label of a CNAME record) may,
   if DNSSEC is in use, have SIG, NXT, and KEY RRs, but may have no
   other data.  That is, for any label in the DNS (any domain name)
   exactly one of the following is true:

     + one CNAME record exists, optionally accompanied by SIG, NXT, and
       KEY RRs,
     + one or more records exist, none being CNAME records,
     + the name exists, but has no associated RRs of any type,
     + the name does not exist at all.

It is confusing, but the difference is that a CNAME is cacheable for queries all types. For example, let's say I query the resolver for qname=example.com qtype=AAAA. The resolver finds the CNAME from example.com to foo.example.org and caches that internally. It then descends a level in its recursion, and finds the answer for qname=foo.example.org qtype=AAAA.

But if the next query is say "qname=example.com qtype=MX", the CNAME is still cached and this knowledge can be used to skip straight to the qname=foo.example.org qtype=MX stage; there's no need to ask the nameservers for example.com anything, since R knows that that name is a CNAME to foo.example.org. But of course foo.example.org doesn't have an MX record, and so things break.

This is the why the order matters, and what I meant by cache poisoning.

So what you're saying is, resolvers will only look up and cache the one type they were asked to query, instead of caching all types associated with the label? While at the same time later dropping all record types if at some point it finally looks up and caches a CNAME? Sounds like a race condition to me!

In general resolvers look up one type and cache the result for that one type. CNAMEs are special; firstly clients never lookup CNAMEs, it's never the qtype. A CNAME is a special kind of answer that says "This name should be considered equivalent to this other name, go find out what that other name has" and as-such the CNAME is cacheable for queries of all types.

It is a race condition, but this race condition only exists if you ignore the DNS RFCs and try to get behavior that DNS just plain doesn't support. I don't mean to point fingers, but in this case that's what happened; CloudFlare's behavior ignores the RFCs [1] and leads to nondeterministic results.

[1] http://blog.cloudflare.com/zone-apex-naked-domain-root-domai...

For those curious to see CloudFlare's CNAME flattening in action (it's seamless, so it's not sexy to look at though): https://gist.github.com/xxdesmus/9202356

root and WWW for shiftyplatypus.com are both CNAMEs (pointing at Heroku).

Reply on news.ycombinator.com