Rsync.net – Cloud Storage for Offsite Backups
rsync.net
rsync.net
1. It is more expensive than S3/Glacier, but they offer "free" read-only snapshots, which can be very useful if your backup software or script does not perform its own snapshots.
2. It can be difficult to determine how much space is taken by the peculiarities of ZFS vs. your own data.
3. Although they offer subaccounts, it is not possible to know the disk usage of a particular subaccount.
4. If you have transfer speed issues, you can contact them and they may whitelist your IP (good if you are doing server backup, not useful for home backup on dynamic IP), which significantly increased speed for us.
5. They have a 10% "free" soft quota, mainly because it is difficult for mere mortals to know in advance how much space your backup will take on ZFS. They send an email when you reach the soft and hard quota.
6. You can communicate with a knowledgeable human by email. This has been helpful to diagnose small but weird issues we were facing.
7. Although we like S3 and Cloudfront, it's nice to have a backup location that relies on standard tool (rsync) and that is outside our usual providers/datacenters.
I really need to say two things here ...
First, the zfs snapshots that are enabled on all normal accounts[1] can be your entire retention scheme. Which is to say, you can just do a "dumb" rsync to us ... no retention, no schedule, nothing - just an rsync every night - and on our end, we handle creating, maintaining and destroying your schedule[2] of full clones of your entire account. It's just like apples time machine, although more efficient since its using ZFS (bits) and not hardlinks (files).
Second, as the parent implies, they are immutable. They are read-only and cannot be destroyed even by our local root. This means that if an attacker destroys your local data, and then gains control of your rsync.net credentials and wipes out your entire rsync.net account, the snapshots will still be there. You can't remove them.
[1] But not on the discounted borg accounts, since we assume smart folks using borg set up their own retention schedule and that is how we make the accounts discounted - by not adding/rotating the snaps.
[2] Yes, you can define whatever snapshot schedule you like - the 7 daily (<1TB accounts) or 7 daily + 4 weekly (>1TB accounts) are the default, but you can add whatever additional snapshots you like (more dailies, weeklies, monthlies, quarterlies ...)
The problem I have with storage providers being expensive is a sense of unfairness. Disk prices get cheaper all the time, so a provider that doesn't periodically cut its prices feels to me like it's been getting progressively more and more expensive. Now that I saw that Backblaze is ten times cheaper than my rsync.net plan, I'm sad :(
Highly recommended.
0.03 USD instead of 0.08 USD. Beating even their top tier -- so long as you follow the constraints mentioned within the link.
Worth nothing that rsync.net pricing includes transfer: "No other costs. No contracts. All transfer/bandwidth/usage is free.".
Looking at https://www.backblaze.com/b2/cloud-storage-providers.html - all services charge more than 3 cents/gb if you actually need to restore from backup.
Now, that doesn't mean rsync.net isn't expensive for many use-cases -- but not as bad as the up front cost difference appears.
Borg 1.2 (~end of 2017, early 2018 perhaps -- wild guess) will include pluggable transfer/storage backends (certainty >98 %).
> - filesystem snapshots are disabled since you'll be doing versioning and retention with attic or borg.
So this means you actually aren't protected from someone taking over your backup server and running rm -rf * on your rsync account.
While their pricing is substantially more than many others (the primary complaint that I see here), in return their support is the best I have ever received from any company ever. Getting responses, in less than ten minutes in some cases, from fully-clued and super-competent engineers - I'll take it.
(no connection to rsync.net other than that I've used them at work in the past and I'm renting space on someone else's account for my personal backups)
I can confirm that nobody that works at rsync.net is not an actual UNIX engineer. You will never talk to anyone here that isn't an expert.
Rare these days, but sometimes you'll even talk to me.
Ah, greetings rsync.net person, and thank you for the reply :) That makes sense. Your staff are a true credit to your company. And the only times I (or people I am working with) have had to talk to you is because we were trying to do something weird like getting a MegaFuck3000 NAS with Debian version -1.5 working with your systems, not because your infrastructure has ever failed. Much love for your company and all that make it as great as it is.
<small>(but seriously I preferred your old webpage)</small>
For the love of god startups should at least test Andoid 2 versions back and iOS current version. It's just sloppy not to be test your home page on mobile.
Let me read the text where I want to put it dammit!
I might try a brand new (attic) account and see if that lands me on a storage array with good performance.
S3 is a great solution and we have nothing negative to say about it. I will note that our special 'borg' pricing gets you down to 3 cents, and we charge nothing for usage/bandwidth/transfer, so pricing-wise, we're about the same.
The main thing that differs (and this is important to some people) between S3 and rsync.net is that rsync.net gives you an honest to god blank unix filesystem to do what you please with.
So while there are many very nice tools that target S3 natively, you can't do magical things like this:
pg_dump -U postgres db | ssh user@rsync.net "dd of=db_dump"
ssh user@rsync.net "git clone git://github.com/freebsd/freebsd.git freebsd"
ssh user@rsync.net "test -f fileThatExists"
ssh user@rsync.net du -Ahd2 some/directory
Although it's worth noting that we do maintain 's3cmd' in our environment, so you can do things like this: ssh user@rsync.net s3cmd get s3://rsync/mscdex.exehttps://www.stavros.io/posts/holy-grail-backups/
About your S3 question, having something like rsync.net (where you can run a server process) is much faster/easier/better because your backup software can know exactly what's stored on the remote side and only send diffs.
I'm sure this can be overcome with some clever client-side engineering, but having a server process is always going to be more flexible overall.
A more pragmatic reason is "because Borg doesn't run as well on S3".
Rsync.net promises exactly what I want, but at 6/gb ($60/mo for 1 tb) it's 5x more expensive than Dropbox so I'm manually hacking around Dropbox's offering for the moment.
https://www.rsync.net/signup/signup_offer.html?code=f0a35f
If you do, however, want an rsync.net account with full snapshotting enabled, etc., you can use our long-standing HN-readers discount, which is currently 4 cents:
https://www.rsync.net/signup/signup_offer.html?code=710b50
I hope that helps - we'd be very happy to serve you with our product.
Single drive, which may limit the use case, but wow, that's cheap.
Even though you are using ZFS on your end, it's possible that it would be cheaper and better to use our 'borg' functionality and send us your data with borg.
Cheaper, since our borg accounts (normal rsync.net account, but with our snapshot rotation disabled - you handle retention on your own, with borg) are discounted down to 3 cents.[1]
Better because the resulting backups are encrypted at rest in a way that rsync.net cannot access in any way. It's just garbage to us.
That's for Tarsnap's encoded data---but that seems roughly comparable to rsync.net's free snapshots. So Tarsnap is something like 5x the cost.
For that 5x cost, you get secrecy of your data (but not your connectivity, metadata, backup schedule) from your backup provider.
However, a four hour restore time (for me) meant I rarely tested the backups on a practical level--since four hours is a long time. It also invalidates some use cases. For example, I'd be at work and want to restore a file I deleted yesterday. My on-site backups (at home) aren't available and it'll take four hours to get that file from Glacier. If that's the case, I'd rather just have it on S3.
So I did S3 for a while, but Google Nearline ended up being cheaper, so I switched to that. Arq supports all of them so it wasn't a big deal. Now I get a bill every month from Google for around $3.62, and can restore files nearly instantly if I need them. It does do incremental backups, although I think it's on the file level. I don't think it's "smart" enough to only upload the changed parts of a large VM snapshot, for example. I just don't have it back up my VMs because mine are all reproducible anyway.
This thread is the first I'm hearing of Borg, so that looks neat. No experience with it, though.
[0] https://www.arqbackup.com/ [1] http://i.imgur.com/Le3bGMb.png
The errors seem to be entirely the fault of OneDrive or ACD but it leaves me wondering what files are backed up (apparently Arq doesn't re-try backing up whichever files were affected until the next backup session), and what the integrity is of files that do exist at the destination.
I'm also unclear on whether any checksum validation is done by Arq (or even can be done) on uploaded files.
Arq does have a button for validating data manually and a separate option for each backup destination that specifies how often (in days) backups should be validated. Whether that's a checksum or not, I don't know.
If there are transient errors it should be retrying. Please send mail to support@arqbackup.com so we can diagnose the problem and fix it. (I work for Arq)
Killer feature: the ability to search images by face, metadata, and KEYWORDS. I can search for "dog" or "server rack" and Photos will return pictures that contain those things. It's pretty accurate.
But if you're using it as an archive, and not so much as a backup (which for me implies restore), then it might make sense for you. There is a simplified calculator here: http://liangzan.net/aws-glacier-calculator/
https://www.stavros.io/posts/holy-grail-backups/
I pay around $50/yr for 150 GB.
e.g. https://www.amazon.co.uk/clouddrive/primephoto or https://www.amazon.de/clouddrive/learnmore/
[](People at /r/datahoarder have petabytes of crap uploaded...)
With Crashplan Family: 160 per year. With the cheapest rsync.net: 1500 dollar's per year.
Yeah, not in 1500 years.
So if you want single machine backup - crashplan is great. If you want long term service data backup - rsync is great.
$ echo $(( (8 * 999) / 100 ))
79
79 dollars/monthI have a home made solution for personal disaster recovery, which is 30€/month in Hetzner (2TB storage, no bandwidth limit) + 6€/month in OVH (same capacity, less CPU and ram, but same storage/bandwidth conditions)...
For me the difference, is not only in the price, but in who has "root access" over the data. Remote backups in third party storage, by definition, should be encrypted. Using rsync almost implies that the remote will be unencrypted (or efficiency will be lost). But we're in the age of "Virtual PRIVATE network THIRD PARTY services" nonsense... so it's hard that nowadays operators understand those principles.
Not sure if you meant the protocol or the service rsync, but neither one implies you store unencrypted data.
If you're going to encrypt all the backup locally before send it to rsync.net, tomorrow you will need to transfer all the GB again, even if just one bit did change...
Of course, you can’t rsync to S3 so I’m handling backups with duplicity and using S3 as the storage target.
Thankfully I’ve not had to perform a complete restore, though I do regular test restores of a part of the backup just to make sure it’s all still working.
Edit: It’s actually $100 (as transfer from S3->Internet adds $0.09/GB)
That said, as others have mentioned, b2's value differentiator is in cold storage. For us, it's largely insurance against failure of our other backup modalities. If you are pulling a lot of data back on a regular basis, the pricepoint is much more in line with the top tier providers.
Note that I for now tend to mirror data between servers/vps' I control/rent, so I haven't really used b2 in anger.
However if you skip that an pay as you go you typically get much more control, much more compatibility (including linux), and much better performance. Like say using S3 or Backblaze's B2.
Or with something like Borg.
(I cannot verify these claims since I have never used b2, I'm just noting what people told me).
Rsync.net has been around forever - like 15 years. They predate the concept of 'cloud'. They were the first to use a 'warrant canary', in 2006. (https://en.wikipedia.org/wiki/Warrant_canary)
It's the minimal legal hack: completely useless in a court but massive generator of internet comments
Though it generating discussion around the warrants themselves is a good feature
They can sure sue the hell out of you after you stop updating though.
- The government cannot force you to update the canary. A court cannot get you to update it, because it's forced speech to demand an update.
- You created the canary of your own accord, and are responsible for its effects. Not updating the canary is, effectively, speech.
Though, from [0]:
"Realistically, though, courts compel speech all the time. Court-ordered apologies, disclosures, and notices are not unusual. And if ever a court would be inclined to compel speech, it would be in a situation like this one, where a company intentionally set out to get around a gag order with this kind of convoluted sea-lawyering."
[0]:http://law.stackexchange.com/questions/268/is-there-any-lega...
http://www.rsync.net/resources/index.html
"Unlimited Support From Real Engineers: support@rsync.net - +1-619-819-9156"
The address is in TOS http://www.rsync.net/resources/notices/tos.html
And more and more companies these days don't even have any fixed physical offices, with remote-first cultures. What's the address of Trello? You won't find it on trello.com
Various aspects of rsync.net are periodically discussed on HN. About 417 days ago I raised a similar issue to yours, namely who is behind this company and what is the address.
https://news.ycombinator.com/item?id=10674029
At that time the owner responded on HN with clear information. It's unfortunate that, more than a year later, their website is still a bit of a dog's breakfast on this issue.
Founder is John Kozubik, who was previously the founder of JohnCompanies - the first VPS provider.[1]
Address is 524 San Anselmo Ave., San Anselmo, CA 94960.
Phone number is +1 619 819 9156.
http://www.rsync.net/resources/index.html
[1] Began offering FreeBSD jails in Aug/Sept 2001. Nobody had coined the term 'VPS' - we called them "server instances".
Again, we had all of this in place prior to an ill-advised website redesign.
As I said higher up in this thread, I'll make sure this gets republished and put in place properly today and I will add some verbiage that addresses your specific point about HQ and jurisdiction.
In the meantime, I will confirm we are a US firm with locations (ie., our own racks - not rented hardware) in San Diego, Fremont (San Francisco, basically), Denver, Zurich and Hong Kong.