Some .io nameservers are returning wrong results again

I Contacted Cloudflare and the response was:

"Unfortunately this stops DNS traffic before it gets to Cloudflare so it is not a problem we can solve for you as much as we would love to. Our recommendation right now would be to contact your domain registrar to understand more about this problem."

Contacted the registrar (iwantmyname.com) and their response:

"You're using external CloudFlare hosting, so our DNS management isn't being used. We don't have any access to the hosting account or the ability to address issues there. It does appear there's some issue with propagation."

Update from registrar:

"It sounds like it's a registry issue. The .IO registry was returning bad data for some of its nameservers. However our upstream partner said that that seems to have been resolved in the last hour or so."

If you DNSSEC sign your domain then the NXDOMAIN would be rejected without associated NSEC records. .io has a DS record in the root-zone. At the very least caches would only be populated with the correct answer.

This is of course assuming the lookups are done using a validating resolver.

We're copping the same at https://elev.io as well...

Timing couldn't be worse, we did a product release on ProductHunt earlier today :(

I am curious why you need a product hunt launch when you have over 500 companies using you? Are they all paying customers?

I think they want more customers.

You should be aware how sketch some ccTLDs are and it would do you some good at this early stage to have a fallback domain.

Afilias manages the .io domain. Give them a call: +1 866.368.4636 https://afilias.info/contact-us/support

It looks like they manage a lot of other domains, all of which should be avoided. Does anybody have a canonical list? It would be silly to switch away from .io to another of their domains!

Not official, but https://en.wikipedia.org/wiki/Afilias

Registry operator: .info, .mobi .pro

Service provider for registry operators of: .org, .ngo, .lgbt, .asia, .aero,

Provider of domain name registry services for countries: .MN (Mongolia), .AG (Antigua and Barbuda), .BM (Bermuda), .BZ (Belize), .GI (Gibraltar), .IN (India), .ME (Montenegro), .SC (the Seychelles), and .VC (St. Vincent and the Grenadines).

Provided ancillary support to: .SG (Singapore) and .HN (Honduras).

You can usually tell from the NS and SOA records. For example, "dig io ns", "dig io soa", "dig org ns", "dig org soa".

Afilias is the only registry operator i know of that uses nameservers with names like "a0.example.com", and the SOA email address has their name.

TLDs can, and occasionally do, switch from one registry operator to another, though. (After all, that's what .io did.)

Afilias also manages the .org domain? Does this mean that we should consider it potentially unreliable too?

Isn't Internet Computer Bureau the administrator? https://www.iana.org/domains/root/db/io.html

Nope, ICB sold their .io registry interest to Afilias a few months ago.

https://www.namepros.com/threads/io-is-transitioning-to-affi...

Perhaps it's still running on the old platform?

.io seems to always have problems. its really sad that lots of people use this domain because it looks cool :( [at work we use this as well]

If this is affecting you it's best to put a long TTL (at least 1 hour) on your domains. That way a good lookup will be cached longer and it will affect less people.

Won't this also cache a bad record for longer like NXDOMAIN which is what we're getting?

No, the TTL for caching NXDOMAIN is decided by the nameservers SOA record. In this case 900 (15 minutes). The reason for this is that if there's no domain there's also no way to get it's TTL.

Thanks for the clarification - much appreciated!

i think negative results are cached using a different value. [https://serverfault.com/questions/426807/how-long-does-negat...]

but last time io had a problem we continued to have downtime after it was resolved because of a high value for the SOA record :(

Thanks for the reply. Much appreciated.

All of my .io DNS records have a TTL of 24 hours and 3600 (refresh-time) 900 (retry-time) 1814400 (expire-time) 60 (negative caching TTL). My PagerDuty is going bat nuts.

If the domain/zone records are handled by someone other than nic-io (say Route53), does that really help?

If I change the TTL for an A-record in Route53, how will that help if there is an NS-issue with nic-io?

(awesome if you could explain!)

Also, does the SOA and NS records have any effect here?

If their servers were unresponsive, then a long TTL might help as caching resolvers would retry and eventually get a successful result. Based on the output of the OPs tests, they are responding with incorrect NXDOMAIN results. This means that resolvers will cache the NXDOMAIN result using the SOA negative result TTL.

You could remove the delegation entries for the name servers that are returning results though, that might help.

nic-io would need to remove the delegation entries, since it is the .io servers that are returning the wrong response. This has nothing to do with any DNS servers actually hosting the <domain>.io zones.

There's a random chance that a DNS request resolves correctly because not all nic-io domain servers are returning incorrect results. If you provide a high TTL on your own records they will be cached for a longer time. Which has as a result that there will be less requests going to the nic-io servers.

This still doesn't rule out bad responses. As far as I know there's no way to achieve that. However, it will make sure correct ones stay active longer.

A high TTL on your NS records would be good as well for the same reason. On the SOA it shouldn't really matter.

Confirmed ns-a2.io and ns-a4.io don't seem to be responding with DNS results.

Customers are already filling up our support inbox, so same here. They can access nic.io

Any quick fix ideas I can get to a customer without technical command-line experience? Changing DNS or modifying hosts file is not really an option.

They boned :-/

In this case, think of it as if the guy who made your maps is giving you the wrong country's maps. It's well made, clear, concise and wrong. There's no 'greater' authority then the map maker, so unless you can reconfigure your users to believe in a different map (change hosts/dns), there's nothing that can be done.

Patience :(

The nameservers seem to be returning correct results for our domain again :) (getstream.io). However, the issue is still there for crates.io, so this seems the outage is still partially ongoing.

Indeed I noticed the same issue for my customers using .io domains (and me unfortunately): https://twitter.com/updownio/status/910508435720065024

Seems to be getting better now. High TTL helps but mostly if it was there before so do it now for the next downtime (it's frequent with .io)

The problem is not actually downtime, because DNS handles that automatically if at least one nameserver is up. The problem is nameservers returning incorrect results. But yeah, .io has quite frequent problems.

Yeah for a DNS server returning wrong results is basically a downtime to me

We're seeing this as well.

Edit: it has now taken out our CI infrastructure and dev environments as it's caching NXDOMAIN records.

It appears that the issue at first impacted all servers in the anycast pool however eventually it only impacted servers ns-a2 and ns-a4. Those servers started returning NXDOMAINs. I am wondering if this was related to the root server key change yesterday. .IO seems to struggle with basic DNS engineering.

I guess that's just the cost of business when your start up is located in the British Indian Ocean Territory

I wonder if they messed up again and let someone register the .io nameservers

https://thehackerblog.com/the-io-error-taking-control-of-all...

Wanted to confirm same issue for my .io domains. This has gotten a lot of people up this morning.

For the curious, the first failure I saw was at 13:15UTC and the last was 14:59UTC.

13:14 UTC through 15:54 UTC here.

Same issue here. Does NIC.io have a status page for their root servers somewhere?

We are getting some client reports of connectivity issues with clara.io. I suspect it is this issue but haven't been able to confirm.

IO apears to have nameservers operated by Afilias, CommunityDNS, OneWorld DNS (which appear to also be CommunityDNS)

    $ for ns in $(dig +short io. IN NS | sort); do
      ip=$(dig +short ${ns} IN A);
      rev=$(host ${ip} | awk -Fpointer '{print $2}');
      echo -n "= $ns = $ip = $rev = ";
      whois -h whois.ripe.net ${ip} | awk -F: '/^descr:/ && !/RIPE/ {print $2}';
    done
    
    = a0.nic.io. = 65.22.160.17 =  a0.nic.payu. = Afilias NS087 A0
    = b0.nic.io. = 65.22.161.17 =  b0.nic.payu. = Afilias NS087 B0
    = c0.nic.io. = 65.22.162.17 =  c0.nic.payu. = Afilias NS087 C0
    = ns-a1.io. = 194.0.1.1 =  ns1.communitydns.net. = ICB plc Anycast (via NetConnex AS21396), ICB plc Anycast (via Community DNS AS42909)
    = ns-a2.io. = 194.0.2.1 =  ns-a2.io. = ICB plc Anycast (via Community DNS AS42909)
    = ns-a3.io. = 74.116.178.1 =  b3-1.oneworlddns.net. = Community DNS
    = ns-a4.io. = 74.116.179.1 =  b4-1.oneworlddns.net. = Community DNS

It looks like a couple of the CommunityDNS nameservers are carrying a different version of the zone than the others:

    dkmpb:~ dknight$ dig -4 +nss io.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 65.22.162.17 in 38 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256292 10800 3600 2764800 900 from server 65.22.160.17 in 44 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900 from server 194.0.2.1 in 62 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 194.0.1.1 in 64 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 65.22.161.17 in 98 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 74.116.178.1 in 101 ms.
    SOA a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900 from server 74.116.179.1 in 107 ms.

And a couple of them are giving bad response

    echo = Afilias
    dig +noall +norecur +cmd +answer +authority -4 +notcp @a0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    dig +noall +norecur +cmd +answer +authority -4 +notcp @b0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    dig +noall +norecur +cmd +answer +authority -4 +notcp @c0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    echo
    echo = Community DNS
    dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a1.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a2.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a3.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS 
    dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a4.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    
    ====== Afilias ======
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @a0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256290 10800 3600 2764800 900
    io.			86400	IN	NS	ns-a3.io.
    io.			86400	IN	NS	b0.nic.io.
    io.			86400	IN	NS	ns-a1.io.
    io.			86400	IN	NS	ns-a4.io.
    io.			86400	IN	NS	ns-a2.io.
    io.			86400	IN	NS	a0.nic.io.
    io.			86400	IN	NS	c0.nic.io.
    hostname.bind.		0	CH	TXT	"ns087b.app22.iad1.afilias-nst.info"
    hostname.bind.		0	CH	NS	hostname.bind.
    shl.io.			86400	IN	NS	ns2.p23.dynect.net.
    shl.io.			86400	IN	NS	ns1.p23.dynect.net.
    shl.io.			86400	IN	NS	ns3.p23.dynect.net.
    shl.io.			86400	IN	NS	ns4.p23.dynect.net.
    shl.io.			86400	IN	NS	ns1.d-zone.ca.
    shl.io.			86400	IN	NS	ns2.d-zone.ca.
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @b0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256290 10800 3600 2764800 900
    io.			86400	IN	NS	ns-a1.io.
    io.			86400	IN	NS	c0.nic.io.
    io.			86400	IN	NS	ns-a3.io.
    io.			86400	IN	NS	a0.nic.io.
    io.			86400	IN	NS	ns-a2.io.
    io.			86400	IN	NS	ns-a4.io.
    io.			86400	IN	NS	b0.nic.io.
    hostname.bind.		0	CH	TXT	"ns087b.app12.lax1.afilias-nst.info"
    hostname.bind.		0	CH	NS	hostname.bind.
    shl.io.			86400	IN	NS	ns2.d-zone.ca.
    shl.io.			86400	IN	NS	ns1.p23.dynect.net.
    shl.io.			86400	IN	NS	ns1.d-zone.ca.
    shl.io.			86400	IN	NS	ns4.p23.dynect.net.
    shl.io.			86400	IN	NS	ns2.p23.dynect.net.
    shl.io.			86400	IN	NS	ns3.p23.dynect.net.
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @c0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256290 10800 3600 2764800 900
    io.			86400	IN	NS	ns-a1.io.
    io.			86400	IN	NS	c0.nic.io.
    io.			86400	IN	NS	ns-a4.io.
    io.			86400	IN	NS	ns-a2.io.
    io.			86400	IN	NS	ns-a3.io.
    io.			86400	IN	NS	b0.nic.io.
    io.			86400	IN	NS	a0.nic.io.
    hostname.bind.		0	CH	TXT	"ns087b.app25.iad1.afilias-nst.info"
    hostname.bind.		0	CH	NS	hostname.bind.
    shl.io.			86400	IN	NS	ns4.p23.dynect.net.
    shl.io.			86400	IN	NS	ns2.d-zone.ca.
    shl.io.			86400	IN	NS	ns2.p23.dynect.net.
    shl.io.			86400	IN	NS	ns1.d-zone.ca.
    shl.io.			86400	IN	NS	ns1.p23.dynect.net.
    shl.io.			86400	IN	NS	ns3.p23.dynect.net.

    ====== Community DNS ======
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a1.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256290 10800 3600 2764800 900
    hostname.bind.		0	CH	TXT	"NORM 002590-E2B186"
    shl.io.			86400	IN	NS	ns1.d-zone.ca.
    shl.io.			86400	IN	NS	ns2.d-zone.ca.
    shl.io.			86400	IN	NS	ns1.p23.dynect.net.
    shl.io.			86400	IN	NS	ns2.p23.dynect.net.
    shl.io.			86400	IN	NS	ns3.p23.dynect.net.
    shl.io.			86400	IN	NS	ns4.p23.dynect.net.
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a2.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900
    hostname.bind.		0	CH	TXT	"COMM 002590-E2B186"
    io.			900	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a3.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256290 10800 3600 2764800 900
    hostname.bind.		0	CH	TXT	"NORM 001B24-2DB609"
    shl.io.			86400	IN	NS	ns1.d-zone.ca.
    shl.io.			86400	IN	NS	ns2.d-zone.ca.
    shl.io.			86400	IN	NS	ns1.p23.dynect.net.
    shl.io.			86400	IN	NS	ns2.p23.dynect.net.
    shl.io.			86400	IN	NS	ns3.p23.dynect.net.
    shl.io.			86400	IN	NS	ns4.p23.dynect.net.
    
    ; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a4.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS
    ; (1 server found)
    ;; global options: +cmd
    io.			86400	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900
    hostname.bind.		0	CH	TXT	"COMM 001B24-2DB609"
    io.			900	IN	SOA	a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900

FWIW, "dig -x" does reverse lookups. Or you can use "ptr" record type and the .in-addr.arpa "FQDN". No need to switch to "host" for your reverse lookups.

I tried loading redis.io and got the network error. I thought it was just their site but looks like a bigger issue with .io.

This affected my service as well. Where does the root of the problem lie, with the .IO service itself or somewhere else?

Yup, we're also seeing this fail for Elastic Cloud (our ES provider which resolves to nodes on aws.found.io).

It affects also our website.

Is there any example of other TLDs having similar issues in past?

Not the first time .io has had issues: https://www.theregister.co.uk/2017/07/10/io_hijacking_in_tra...

We were affected as well, but are back on ns-a[24].io. since about 1550Z.

You know, it's ironic, but https://dnsspy.io - which also uses the .io TLD - would alert whenever something like this happens, so you don't have to spend hours debugging a DNS issue.

Ours (kasko.io) seems to be working again on both ns-a2 and ns-a4

Incomplete copy pasta, or your dig commands are off?

Yup, seeing this as well for one of our .io domains.

Also seeing this affect several of our .io domains.

Same here. Started 14:20 UK time (13:20 UTC).

.io had a similar issue beginning this year.

same issue here

We're getting this as well.

it's interesting, last night I was trying to get to sci-hub.io and all that showed up was a blank page. I assumed due to the nature of the site that maybe it was taken down but instead it's a .io problem

Seems to be fixed now

For us too, but not everyone. Seems like they're rolling out some kind of fix.

Seeing the same issue

Having same issues

IO apears to have nameservers operated by Afilias, CommunityDNS, OneWorld DNS

$ for ns in $(dig +short io. IN NS | sort); do ip=$(dig +short ${ns} IN A); rev=$(host ${ip} | awk -Fpointer '{print $2}'); echo -n "= $ns = $ip = $rev = "; whois -h whois.ripe.net ${ip} | awk -F: '/^descr:/ && !/RIPE/ {print $2}'; done

= a0.nic.io. = 65.22.160.17 = a0.nic.payu. = Afilias NS087 A0 = b0.nic.io. = 65.22.161.17 = b0.nic.payu. = Afilias NS087 B0 = c0.nic.io. = 65.22.162.17 = c0.nic.payu. = Afilias NS087 C0 = ns-a1.io. = 194.0.1.1 = ns1.communitydns.net. = ICB plc Anycast (via NetConnex AS21396), ICB plc Anycast (via Community DNS AS42909) = ns-a2.io. = 194.0.2.1 = ns-a2.io. = ICB plc Anycast (via Community DNS AS42909) = ns-a3.io. = 74.116.178.1 = b3-1.oneworlddns.net. = Community DNS = ns-a4.io. = 74.116.179.1 = b4-1.oneworlddns.net. = Community DNS

It looks like a couple of the CommunityDNS nameservers are carrying a different version of the zone than the others:

dkmpb:~ dknight$ dig -4 +nss io. SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 65.22.162.17 in 38 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256292 10800 3600 2764800 900 from server 65.22.160.17 in 44 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900 from server 194.0.2.1 in 62 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 194.0.1.1 in 64 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 65.22.161.17 in 98 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256293 10800 3600 2764800 900 from server 74.116.178.1 in 101 ms. SOA a0.nic.io. noc.afilias-nst.info. 1497256201 10800 3600 2764800 900 from server 74.116.179.1 in 107 ms.

And they're giving bad responses

echo = Afilias dig +noall +norecur +cmd +answer +authority -4 +notcp @a0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS dig +noall +norecur +cmd +answer +authority -4 +notcp @b0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS dig +noall +norecur +cmd +answer +authority -4 +notcp @c0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS echo

echo = Community DNS dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a1.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a2.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a3.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS dig +noall +norecur +cmd +answer +authority -4 +notcp @ns-a4.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS echo

= Afilias

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @a0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 io. 86400 IN NS c0.nic.io. io. 86400 IN NS ns-a4.io. io. 86400 IN NS b0.nic.io. io. 86400 IN NS a0.nic.io. io. 86400 IN NS ns-a3.io. io. 86400 IN NS ns-a1.io. io. 86400 IN NS ns-a2.io. hostname.bind. 0 CH TXT "ns087b.app16.iad1.afilias-nst.info" hostname.bind. 0 CH NS hostname.bind. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns1.d-zone.ca. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net.

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @b0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 io. 86400 IN NS a0.nic.io. io. 86400 IN NS b0.nic.io. io. 86400 IN NS c0.nic.io. io. 86400 IN NS ns-a1.io. io. 86400 IN NS ns-a2.io. io. 86400 IN NS ns-a3.io. io. 86400 IN NS ns-a4.io. hostname.bind. 0 CH TXT "ns087b.app10.lax1.afilias-nst.info" hostname.bind. 0 CH NS hostname.bind. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net. shl.io. 86400 IN NS ns1.d-zone.ca.

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @c0.nic.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 io. 86400 IN NS ns-a2.io. io. 86400 IN NS c0.nic.io. io. 86400 IN NS ns-a1.io. io. 86400 IN NS a0.nic.io. io. 86400 IN NS ns-a3.io. io. 86400 IN NS ns-a4.io. io. 86400 IN NS b0.nic.io. hostname.bind. 0 CH TXT "ns087b.app9.iad1.afilias-nst.info" hostname.bind. 0 CH NS hostname.bind. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns1.d-zone.ca.

= Community DNS

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a1.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 hostname.bind. 0 CH TXT "NORM 002590-E2B186" shl.io. 86400 IN NS ns1.d-zone.ca. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net.

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a2.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 hostname.bind. 0 CH TXT "COMM 002590-E2B186" shl.io. 86400 IN NS ns1.d-zone.ca. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net. = OneWorldDNS / CommunityDNS?

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a3.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 hostname.bind. 0 CH TXT "NORM 001B24-2DB609" shl.io. 86400 IN NS ns1.d-zone.ca. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net.

; <<>> DiG 9.8.3-P1 <<>> +noall +norecur +cmd +answer +authority -4 +notcp @ns-a4.io. io. IN SOA hostname.bind. CH TXT shl.io. IN NS ; (1 server found) ;; global options: +cmd io. 86400 IN SOA a0.nic.io. noc.afilias-nst.info. 1497256301 10800 3600 2764800 900 hostname.bind. 0 CH TXT "COMM 001B24-2DB609" shl.io. 86400 IN NS ns1.d-zone.ca. shl.io. 86400 IN NS ns2.d-zone.ca. shl.io. 86400 IN NS ns1.p23.dynect.net. shl.io. 86400 IN NS ns2.p23.dynect.net. shl.io. 86400 IN NS ns3.p23.dynect.net. shl.io. 86400 IN NS ns4.p23.dynect.net.

Try indenting those lines two spaces.

same

Same issue here! Luckily, all our API clients include a fallback domain.

Luckily? No, that sounds intentional and well-planned.

Might be limited to Route 53: https://status.aws.amazon.com/

I'm not getting that impression from the status update:

"We can confirm that resolution of .io domain names are intermittently failing due to issues with the .io TLD name servers that are hosted external to AWS. Route 53 name servers continue to operate normally, but customers who have sub-domains of .io hosted on Route 53 may experience issues until the .io TLD name servers hosted externally are resolved."

how can that be related to Route53? Nameservers are managed by .io TLD and that's where the issue is coming from. If you use Route53, the nameserver should return Route53 nameservers