Comment by berdario | Hacker News Reader

Comment by berdario | Hacker News Reader

I got a 4 year contract with Crashplan (expiring in 2018) and I'm also pissed by this news...

I paid less than 4$ per month (the price would've been already higher if I renewed yesterday, but still acceptable) and stored around 130 GB of data (so not a lot, but it's already more than the 100GB threshold that some services use)

I need something that would keep watch on all the files on the system (see inotify) to avoid continuous rescans of the disks, and that would work on Linux...

Just these 2 simple requirements already seem to disqualify any other tool suggested here:

Carbonite (no Linux)

Backblaze (no Linux, and none of the 3rd party integrations with B2 seems to support live watching of files)

Tarsnap (no watching)

Borg (no watching)

Does anyone know if it's feasible to keep the whole / under Dropbox, Google Drive or Spideroak? (I know that GDrive doesn't have an official linux client)

The only solutions I know would be:

Lsync (but it would be only syncing, no history), r1soft (but it seems to be only servers, haven't checked yet the features)...

Are there any other tools out there?

I'd really thinking that I should write my own

Brian from Backblaze here.

> Backblaze ... none of the 3rd party integrations with B2 seems to support live watching of files

I'm not exactly sure what you are going for, but you can "preview" files through the web interface on BOTH the Backblaze online backup and also B2.

If you are specifically asking about movies, just stay tuned. The first step was allowing preview of simple images, next we want to support previewing of movies.

Also, a feature we added a week ago in Backblaze Online Backup is the ability to "Share Files" (any file in your backup) with other people. For the first release we limited the file size (just so our servers did not tip over) but we plan to support the ability to share all files less than 10 GBytes.

>I need something that would keep watch on all the files on the system (see inotify) to avoid continuous rescans of the disks, and that would work on Linux...

What op is looking for has nothing to do with previews

By live watching, the commenter is referring to 'watching to changes to a file'.

Just googling "inotify linux backup," I found my way to this post about inotify not being recursive [0], doesn't that limit its usefulness for triggering backups of an entire drive?

For Linux and the relatively small amount of storage you require, you're very likely going to be looking for client software independent of backup storage medium, not a service that provides a client. 130GB would be about $3/month on AWS S3.

[0] https://www.quora.com/Linux-Kernel/Inotify-monitoring-of-dir...

Yes, from my understanding you have to watch on inotify every single file on your system (watching / is not enough), which causes a considerable increase in memory usage for the kernel (like, more than 500MB on my laptop), along with forcing you to increase fs.inotify.max_user_watches

Not ideal, but you only need to do it once and I already have it set up in my ansible playbook:

https://github.com/berdario/dotfiles/blob/5c999113d9ce993f01...

I researched now for a solution that lets me backup all my family PC's and found this two solutions that I will now try out:

https://www.duplicati.com/ https://www.urbackup.org/

edit: It's client/server backup software. Not an online service. I used CrashPlan in a similar manner.

I'm also a Crashplan user. Don't have great suggestions, but in terms of Google Drive on Linux, Insync (https://www.insynchq.com/) seems to work well enough.

Wish Google would just release the Linux client they use internally.

Can you explain what you mean by "watching", and why it's important to you?

> I need something that would keep watch on all the files on the system (see inotify) to avoid continuous rescans of the disks

Inotify (on Linux, and fanotify wouldn't be enough, since the backup tool would have to track file deletions as well) is the only way to have seamless background backups

I tried to use other tools in the past, like duplicity... But having a scan every hour that takes several minutes and has noticeable CPU usage is a deal breaker

(and having a backup scheduled less frequently than that is also a dealbreaker, since you cannot rely that the laptop would be up an running at a certain time during the day)

(Crashplan also does a full rescan, but it's not strictly necessary... It's basically needed if the backup daemon crashed or was shutdown, or stuff had been changed on the filesystem while the OS was off, so that sanity check rescan can actually wait a few days)

I think there are a couple of variants. First is watching for files to change and then uploading them when they do (possibly waiting a short period and collecting a set of changed files before uploading). This is what folder syncing programs like Dropbox do and probably what berdario is seeking.

Second is watching for files to change and saving that list until the next scheduled backup is run. This is what macOS's Time Machine does, the OS's fsevents continuously keeps track of changes and Time Machine just asks it for the list of changed files since a given time (the last time Time Machine ran). This is an alternative to a backup client scanning the filesystem itself on a schedule.

Yes, what I'm actually looking for is the 2nd one. (that is, I don't care that the backup happens now or in 15 minutes, but I care that when the backup runs, the tool doesn't have to rescan the whole filesystem)

Reply on news.ycombinator.com