Anycast TCP

In practice the "split path" scenario is only a problem when topology changes. Routers typically use a hash of source and destination port, source and destination IP address, and sometimes ingress port id to pick which of N equal paths to take.

This is done not just to preserve end destination, but so every sequential packet in a flow follows the same path, thus must arrive in the same order. (Different paths have different latencies because of distance and buffer depth on the routers.) Many tcp implementations will back way off as soon as they see out of order packets, so network providers must do this.

The title is misleading - it makes it sound like this is a description of how anycast works. However, this appears to be a proposal for how anycast could be implemented. But, its not clear to me that it's actually implemented this way in practice.

A big chunk of the proposal seems to involve maintaining state between distributed servers - which is hard to do at all and even harder to scale. There also seems to be some layering violations - if a transaction is going to be too large, this proposes using an HTTP redirect. I might have missed it, but, this is hard to do when working at the TCP layer. The redirect would have to be done to a non-anycast address. Which means that every server needs to know about all the other potential servers that are running.

Interesting! with a few flaws. Out of order quoting:

> An Anycast TCP system can be achieved without any modifications to the TCP protocol.

Yet he describes a system that requires modification.

> With protocols that support redirects (like HTTP) ... use the initial anycast connection to determine which site is closest to the client and then send a redirect instructing the client to connect to a unicast address

> QUIC packets returned from the server to include a unicast address

These techniques already exist, patented by Google. It's not dissimilar from anycast in ATM networks, which is where I believe anycast originated.

> Experiments with Anycast DNS show that in 99%+ cases Anycast devolves to unicast.

Too bad he doesn't say what experiments, but also he should have analyzed from this that the primary (and only practical?) use of anycast is for wide area gslb and HA. So let's not talk about toy implementations and lets talk only about true global scale impact.

At this scale, he fails to acknowledge the difficult of sharing state globally. He should make more of a point that only connections which exceed a the high water mark within their local sequence number space need to have the ownership info flooded. He needs to comment on the need for authentication of this data (otherwise you have the makings of a synflood-like attack). He fails to recognize the difficulty of draining connections and bringing up new nodes, while others still operate and may have to have their sequence number space re-partitioned.

It's too bad he doesn't discuss how well (or poorly) this could be handled with client-side support (like http redirects, but at the TCP layer). Whilst his approach does require stack changes, at least it's only server side.

He seems to think that DSR (as implemented by LVS in his example) performs better than than termination by the load balancer. He's off on his knowledge by 5 years if not a decade.