alyrik

I'm mainly excited that you can just deploy a generic tool like Bacalhau across your cluster and have it as a platform to manage your logs and a whole bunch of other workload/management stuff, rather than needing to install a million different special-purpose things on every machine!

> The author is worried about the overhead of the three-way handshake, but the overhead of setting up TLS is far worse.

Oh, "TLS sucks" is a whole separate blog post for another day ;-)

Performance: A bit meh at bulk bytes-per-second snapshotting, particularly when the vault is accessed over ssh, as it's latency sensitive (send block, wait for response, get next block, send, wait for response, repeat - rather than streaming the blocks). However, the mtime cache means that it's really good at spotting the files that have changed, so it scans my couple of terabytes fairly rapidly and uploads the tens of megabytes that have changed each day.

Mature: Well, it's been running nightly for years for me, and has saved the day on a few occasions when I've had to restore stuff from the vault.

Reliable: The only data I've lost was when a vault disk actually died, and I hadn't replicated it because, well, I was happier losing some history sometimes rather than buying more disk for that use case. However, when I get archive mode in production, I'm going to build a better vault replication solution than rsync!

I aint'n't dead :-D

Largely, Ugarit is ticking along nicely in my production setup, doing backups of my servers; when I get time I work on (a) performance, which is still weak in some cases and (b) archive mode, which is a fun spare-time project rather than something urgent I use. But the core case of doing backups Just Works(tm).

Oh dear... I once logged into the postgresql database of a very busy hosted service in order to manually reset a user's password. So I started to write the query:

UPDATE principals SET password='

Then I went and did all the stuff required to work out the correctly hashed and salted password format, then finally triumphantly pasted it in, followed by '; and newline.

FORGOT THE WHERE CLAUSE.

(Luckily, we had nightly backups as pg_dump files so I could just find the section full of "INSERT INTO principals..." lines and paste in a rename of the old table, the CREATE TABLE from the dump, and all the INSERT INTOs, and it was back in under a minute - short enough that anybody who got a login failure tried again and then it worked, as we didn't get any phonecalls). It was a most upsetting experience for me, however...

A cheap NAS made from a Raspberry Pi model B (the Ethernet one) and an off-the-shelf USB external HDD.

That's a good point - version control and build/release process management are all too often reinvented from scratch after initial painful experiences in the field!

Check out what RainStor does to compress relational data by a factor of 20-40 or so (that's 95%-97.5%): http://www.youtube.com/watch?v=rIsVcaMaAgg&feature=playe...

It's a general question, but the linked article explores it from a few angles of different kinds of career :-)

It agrees with your points - a split into applied and theoretical would probably be a good thing. The programmers I work with in the UK seem to come from physics/mathematics backgrounds, or electronic engineering, with just a few CS graduates. All of them (even the CS graduates) feel that a large proportion of what they learnt on the degree is irrelevant to their career going forwards. A degree focussing on practical software engineering would probably be very useful!

SOCKS gets you end to end (though it's arguably not "real"!) by letting you have listening sockets (which NAT doesn't).

There is indeed a scaling issue with NAT/SOCKS as both need connection state on the gateway device - but, I suspect, the dropping cost of hardware versus the rising cost of IPv4s will favour more hardware. The real question is how it'll stack up against the cost of dropping IPv4 for IPv6 (not the cost of adding IPv6 support alongside IPv4, which isn't that hard).

Yeah, I'd like easier peer-to-peer communications, too.

Which is why I suggested setting up SOCKS as a competitor to NAT, as it lets the client ask the gateway device for an external IP/PORT to do peer-to-peer communications over... without needing to set up IPv6 :-)

I've run IPv6, to try it out... I just didn't bother setting it up again next time I replaced things, as it gained me nothing, so I didn't want to repeat the effort.

So it's in my best interests to wait until I have to.

Which means I'll continue to use IPv4.

Which means the sites I want to connect to aren't motivated to turn off IPv4.

Ad infinitum.

I've seen the "IPv4 will get more and more expensive, and more and more sites will provide IPv6 connectivity, and (for some unspecified reason) clients will put out the effort to get IPv6 connectivity for no near-term benefit, and then when nearly everyone has IPv6 connectivity, IPv4 support will become like IE4 support and sites will start dropping it, and the IPv4-only people will then demand an upgrade" argument, and I think it has flaws.

* Why do clients - home ADSL users, small offices, wifi hotspots - want to bother with IPv6? It offers them no benefit for at least several years. Everything that's good is IPv4 only, or maybe IPv4 and IPv6. All they need is one external IPv4 - or they can share an upstream IPv4 via carrier-grade NAT, so they needn't bear any "rising cost" of IPv4.

* Why should important sites really bother about IPv6? They already have large IPv4 allocations, and there's endless tricks to make better use of them (they can vhost any HTTP-based service, for a start). Moving to IPv6 will make it easier for startups to get IPs to compete with them. I'm not sure why some of them have offered limited IPv6 access (Facebook's is just a proxy that forwards on the connections via IPv4, it seems), but they don't seem to maintain them well (bit.ly was inaccessible via IPv6 all day; nobody seemed to notice) - I suspect they're mainly "20% time" projects.

* Just how near is the point where it's a good idea to drop IPv4, for clients or servers? There's a lot of legacy networks to shift... more so on the client end than on the server end, which is dominated by a "top 100 sites" or so that could all conceivably add IPv6 support with little effort.

Of course, what I'm proposing here - fixing the problems with NAT as a way of making IPv4 address exhaustion not be a problem for clients - doesn't help with IPv4 exhaustion for servers.

What would I propose for that?

* Using them more efficiently by having protocols support endpoint identification within a server (HTTP supports virtual hosting, SSL has spreading support for host identification in the SSL handshake process, SMTP has been fine with it for years, etc).

* Using incoming connection proxies / load balancers to have a small number of external IPs, connections to which are handled by a large number of backend servers

* Perhaps in the longer run, better usage of SRV records so that well-known ports fall into disuse, and server ports can be assigned by the administrator and then placed into the SRV record for that service, in effect making IPv4 addresses be 48 bits long.

I agree... the problem is, how are we to get a single global namespace?

I have serious concerns with IPv6's practicality, even with 6in4 et al. Is trying to implement it just flogging a dead horse?

If somebody comes up with a way to actually get IPv6-only nodes widespread - not just nodes with joint IPv4 and IPv6 machines - then I'll have hope for mainstream IPv6. But while every client and server has to have IPv4 as well to be of any use, then what benefit does having IPv6 connectivity give it?

Interesting point, that... Addresses being scarce and valuable would play into the hands of large ISPs who already have a lot of addresses! Discuss! :-)

Yeah, but what benefit is there as opposed to just sticking with NATv4 or SOCKS?

The real issue here is running out of IPv4s.

We don't really need more IPs for clients - NAT fixes that problem (I just argue that SOCKS could be a better solution, that can work alongside NAT).

We need more IPs for servers. The IPv6 solution is to move clients and servers to IPv6. 6in4 gets around needing to migrate the ISP as well, which is good, but it's still a heap of work that has to be done by people who don't get any material benefit out of it for years, at best. So it won't happen.

Adding SOCKS to NAT boxes and to client devices is easier than IPv6, I warrant; SOCKS is a dead simple protocol, and the client can be slid into the sockets library without the app having to care.

And that's it. There's no need for ISPs to route you IPv6, or to set up tunnels. No need for dual stacks and AAAA records. No need for service providers to provide IPv6 access to their servers.

SOCKS is less restrictive than NAT - the client can know about it and find their external IP/port, it allows incoming connections, it lets the gateway device authenticate users and give them differing quality of service, etc.