Comment by nirvana | Hacker News Reader

Comment by nirvana | Hacker News Reader

Parent Full thread

I see the everything below the new EX6S has dropped by about 90 Euros in setup fee. This is great news! I think I'll buy 6!

People have brought up reliability and that they are using consumer grade hardware. This is an issue if you have SPOF. If you have a fully distributed system (rare these days, for sure) it isn't much of an issue.

My current plan is to use DNS and each box is a full stack. (web app platform on top of riak with authoritative DNS on the box.) So a web request might look up example.com and get back a list of authoritative name servers NS1-6.exampledns.com When the client then does the query to one of those auth servers the auth server is in the cluster and returns the list of other servers in the cluster ranked by load (Eg: multiple A address response for the query.) Then when the client goes to connect to the web server it will hit the least busy node.

I wonder, though, if there are 5 authoritative name servers listed in the root for a given domain, will the root return them in the same order every time, such that my first authoritative dns server (the one listed first at the domains registrar) will get most of the DNS load? Or is there a way to have the root name servers randomize the order of the authoritative servers they give back to the client?

(Yes all this will be open source, eventually. I've learned not to make promises about when-- soon as its viable outside the lab.)

DNS load is typically fairly light, because it's just a few packets per hit. You certainly don't need 6x redundancy, and if one server gets most of the traffic it's probably no biggie.

There are a couple of caveats to your load balancing strategy. With enough headroom, these probably aren't total game breakers, but you should be aware of them. More at http://serverfault.com/questions/60553/why-is-dns-failover-n...

1) You shouldn't expect even or consistent load balancing across servers. Some caching DNS servers (such as those at large ISPs) have very many downstream consumers, and they won't do any randomization. If a large DNS server sees a new order of records, it might trigger a synchronous switch of 10% of your customer base from one server to another. This will cause spiky traffic.

2) You can't rely on any kind of sticky sessions. This may or may not be a problem, and many load balancers drop this guarantee as well for performance reasons, but it is certainly possible that a client may see a DNS records TTL expire and switch to a new IP. If you aren't prepared for that you may start dropping sessions.

DNS doesn't play particularly nice when you try and load balance it. You end up, essentially with issues with end users caching particular ip addresses and either failing when they shouldn't or causing load imbalances on particular servers that you can't seem to fix.

You probably want to have an external dns host returning two ip addresses for a haproxy or LVS cluster, which you then route into your actual web tier.

IIRC the way to get clients to round robin connect to different servers is to have your DNS server(s) return multiple IP addresses for a given domain.

I have no idea about how authoritative name servers work, but I'm assuming it's a prioritized list. I'd probably have all your authoritative servers provide all the IP addresses in any case.

perhaps somebody should put some scripts together and sell them? I wonder how well would CloudFoundry run on this. The biggest concern is the database, since that'd be the SPOF that is hard to handle.

Reply on news.ycombinator.com