Comment by jimaek | Hacker News Reader

Comment by jimaek | Hacker News Reader

Hey everybody, im the creator of dnsperf. I just wanted to clear some things up.

1. The point of the service is to have an objective way to compare different DNS services. dnsperf is supposed only to help you in your search for the best DNS service and not crown the absolute winner.

2. I plan to add as many locations as possible to get more reliable data. More locations will also solve the problem of "DNS at the same datacenter as my test nodes"

3. Some people are confused about the node locations. Here is the map http://www.dnsperf.com/network

4. Its not open source but some time in the future it will be.

5. The tests I am running are supposed to exclude variables such as local/ISP caching, DNS proxies, resolvers, SRTT and so on. I wanted to test the nameservers directly and get the raw performance data of the provider himself. I believe thats the best way to do fair comparison.

Thank you for your comments and love. I'm glad people liked the service :)

Feel free to email me with your feedback or questions.

Hi jimaek!

Thanks for putting this service together. For an African testing location maybe a VPS provider from ZA would work? Following are a few providers with Joburg or Cape Town presences (haven't personally used any though so YMMV):

http://www.vpsnine.com/hosting/locations.php http://www.webafrica.co.za/hosting/vps/ http://www.webnow.co.za/vps/ http://www.web-telecoms.co.za/vps/

(posted these to the comments on your site but re-posting here in case you miss it).

Thank you, I will look into them.

Could you please add the IP or the provider of the client performing the tests? It seems that you are using servers hosted with OVH, this makes the OVH results better because your tests are within the provider network.

I use only 2 servers hosted with OVH. I see the problem here but there is no way to avoid it. Monitoring 37 providers will result in multiple "same DC" problems. I plan to solve this problem by keep adding new locations with different datacenters. The more locations I have the less impact this kind of issues will have.

If you provide the "same DC" information, this helps making the right decision. Maybe you can exclude the "same DC" results or offer an option to filter them out. This of course reduces the number of probes for the given provider, but maybe no probes is better than a too good to be true probe.

This! If you have dc-info, you can ignore intra-dc lookups.

I simply cant know what DCs are used by the providers that I monitor. So I have no idea where we share datacenters. What happens if they change DCs? I would need to constantly monitor their infrastructure. Filtering out would be too hard and pretty useless.

By simply adding more locations I fix the problem, add more data and simplify the whole process.

The word "speed" usually suggests that more is better.

What the graphs actually display is latency - in which case less is better.

That was confusing for a few moments.

I guess I will need to fix that. thanks for the feedback

Latency is a function of the speed of light - and in most engineering realms "speed" is well understood when using latency as a description. I had no qualms understanding what the creator meant, but I agree that the technical reality is the graph should be defined by "query latency" vs "query speed". Either way, fantastic data!

Hopefully this data is trended over daily, weekly, monthly, quarterly, and yearly rollups. I see monthly when drilling into a provider, but that's all. A rolling annual graph would really tell a true story of how consistent a provider is and that, in itself, is very important when considering outsourcing DNS.

This is cool and I've seen it before but I have a hard time interpreting the results. For instance, I have a hard time believing Google's DNS is substantially slower than anyone else's which leads me to question all the results.

Also, and I'm hoping someone here can clear this up for me, but it's not clear to me how these results translate to actual DNS performance for my site. I only vaguely understand how the magic of the internet works but don't the DNS results get distributed throughout the network making the performance of the root servers less relevant? If I could chop 50-100ms off my TTFB by switching my DNS from Namecheap to someone else, I would definitely pay for that, but it's not clear to me.

Why would you pay more for faster DNS? Most browsers come with DNS prefretching built in, I doubt it really matters for a website.

Because the first page load is what matters most.

Your DNS information should already be cached in all major ISPs DNS caches (unless you mean the first page load ever).

How about the first subsequent load after the TTL for the previous load expires?

Admittedly, not much of an issue if you're a big site with hundreds/thousands of requests a minute but it is an issue if you're a less trafficed site and have a low TTL.

Did you read what I wrote? I said I would pay if I were convinced it would lead to faster loads, but I'm not.

Another way of gathering data would be a js-snippet that participating websites can include, which would run a random subset of the tests on some share of of visitors, and report back to your service.

It'll be more realistic then DC -> DC queries.

- http://www.w3.org/TR/resource-timing/

- http://googledevelopers.blogspot.se/2013/12/measuring-networ...

Its realistic but not consistent and reliable. Like I said I want to test their raw performance without any variables in between. All providers get tested directly from the exact same locations and times. Thats the data I wanted to get. It makes it easy and fair to compare.

> It'll be more realistic then DC -> DC queries.

That depends on what you're trying to measure. If you want to know what the total end-user experience is like, Resource Timing is awesome.

If you're trying to compare DNS providers without noise from the differences in client implementations and ISP competency, however, it'll be very noisy and you need a massive number of samples to avoid seeing trends which are really just sampling artifacts.

The best approach would be to do both so you could compare the server response times with the measured client values so you can get an idea whether a slow-down this week is caused by something you have any control over.

Would you mind updating your bar charts to say "Query time" instead of "Query speed". Some mention of the unit of the numbers would be nice as well, such as "Query speed ( ms/req )"

Reply on news.ycombinator.com