Hacker News Reader

Comment by emilburzo | Hacker News Reader

After yesterday's post[1], I've setup a cron job to regularly backup Gmail/Google Drive.

I'm using Gmvault for Gmail (emails and chats), pretty effortless and with enough features for me, especially the export to maildir.

And rclone[2] for Google Drive, in "create a local mirror" mode (sync).

Any other tools to backup your Google life?

[1] - https://news.ycombinator.com/item?id=12972554 [2] - http://rclone.org/

View on HN

I am actively trying to get off of dropbox. I really want a similar native application experience that syncs to-and-from S3. Not a cron job using the s3 cli, not something that only works on osx/windows/etc... So far owncloud enterprise is the only polished looking solution I've found, but thats a bit overkill...

I've been using syncthing and I'm reasonably pleased with it. I don't use it heavily and only over my local network, though, so I'm not sure how well it handles archiving changes and file conflicts.

You may want to have a look at odrive (https://www.odrive.com). Use it to sync a bunch of different storage accounts via this single app. Works alright for what it does, and they have an S3 option.

Won't forklift serve your purpose?

You are looking for Syncany: https://www.syncany.org/

But really, you may not need S3 at all and just sync between your devices with Syncthing: https://syncthing.net/

Shameless plug: Syncthing is in official Debian repositories! https://tracker.debian.org/pkg/syncthing

"The core team of Syncany is on hiatus for an indefinite amount of time. Feel free to do with the code what the license allows and encourages, but please don't expect any maintenance"

Not feasible to use this for critical infrastructure if there is no maintenance happening.

How have you found syncthing perf? When I tried it a while back the perf was horrible. It ate CPU and was pretty slow, even though I was just testing with 2 machines on my local network.

It's still slow and use lots of CPU.

This was improved recently with 14.0.7:

- https://github.com/syncthing/syncthing/releases/tag/v0.14.7

Also, It will improve further with File System Notifications (from what I understand?):

- https://github.com/syncthing/syncthing/pull/2807

Thanks, Syncthing looks like exactly what I was looking for! I'm guessing it needs some external server for coordination? It seems a bit unnecessary to restart after each config chsnge, is there some technical reason for it? Nice job otherwise, will try it out.

Yes, it needs an exernal server.

By default, Syncthing is pre-configured with community-hosted discovery servers:

- https://github.com/syncthing/syncthing/blob/master/lib/confi...

The community also hosts relay servers, so if your two devices can't communicate with eachother directly, it will work anyway.

Relay servers take bandwidth. Anyone can run a relay server, and it will automatically join the relay pool and be available to Syncthing users. This is documented here:

- https://docs.syncthing.net/users/strelaysrv.html#strelaysrv

It would also be possible to host your own private relay and discovery pools, if you need that for some reason.

How about Tarsnap?

Please don't use OwnCloud. It eats your files and cost us loads of time and effort, in addition to sowing FUD among my office coworkers, who thought someone was deleting files from the shared/sync drive. Plus it doesn't support delta sync [0], so if (for example) you're syncing large files like (for example) True/VeraCrypt volumes, you're going to be pushing a lot of data around. This is especially awful since you're not doing this on a LAN but to S3, which means your raw cost in dollars for operating this software will be much much larger than with another solution like SyncThing or Seafile which does support delta sync.

[0] https://owncloud.org/faq/#partialsyncing

ownCloud, or nowadays NextCloud [1], is still the best solution I have, hosted on a private virtual server. It's not a drop-in replacement for sure -- I'm basically prepared to do a full clean reinstall each time I want to upgrade -- but they have desktop and Android clients, and it's working just fine. Of course, I also have a separate backup of all the files.

[1] https://nextcloud.com

I use amazon cloud drive unlimited $60/yr and arq backup to have client side encrypted backups. I also arq backup to my local NAS.

If you want to use an open source tool you can use borg backup & rsync.net as your external backup site. Borg doesn't have good S3 integration, and using fuse & s3 doesn't work that well either. It works best when the borg daemon is on the reciever box too to help with indexing and such.

http://www.rsync.net/products/attic.html

Interesting you mention this ...

There's a big item on one of my whiteboards: "put gmvault into the environment" ... the idea being that you could run 'gmvault', over SSH, on rsync.net:

ssh user@rsync.net gmvault ... blah blah ...

I've been meaning to do this forever ... it would be great if rsync.net customers could not install anything, but just run gmvault as an ssh command.

The only reason it takes time is that we do not have a python interpreter in our environment - we try to keep things as simple and locked down as possible - which means we have to "freeze" gmvault as a binary executable in order to put it into place ...

So ... folks want this ?

I did an extensive lookup of dropbox alternatives last year, and ended up self-hosting seafile. Multiple family members use it and it works great, plus encryption built-in from day one (me not being able to read their stuff is critical to me). There has been no outage, and upgrades are easy.

Reply on news.ycombinator.com