Comment by brianwski | Hacker News Reader

Comment by brianwski | Hacker News Reader

Parent Full thread

Disclaimer: I'm biased because I work at Backblaze. I'm also one of the founders. Backblaze started in my living room in Palo Alto.

Backblaze is A LOT LESS LIKELY to exit out of providing consumer online backup anytime soon as it is our primary business.

I don't fully understand what just occurred with CrashPlan, but they had two separate "clients" - one for consumers and one for businesses. As far as I can tell, CrashPlan is discontinuing one of their two clients but doubling down on the "business client". I don't understand how they got into that situation, but Backblaze only has one backup client so we can't really abandon the one true client we have. :-)

Finally, Backblaze is profoundly different than CrashPlan in that we never really raised any bank financing or VC financing. We're 90% employee owned, and there are no deep pockets. CrashPlan raised something like $150 million which comes with "pressure to grow fast or die". Backblaze is free of any such pressure, we own our own fate.

In a final note: I have ALWAYS liked CrashPlan and I am sad to see them go. Realistically they were never a "competitor" to Backblaze. Our biggest competitor was customer apathy and customers not realizing that online backup was a good option. The more money companies like Carbonite and CrashPlan poured into advertising INCREASED Backblaze sales merely by raising awareness of online backup. CrashPlan (and Carbonite) have been absolutely wonderful to Backblaze because we didn't have the gigantic amounts of money to advertise that they had and they essentially advertise on our behalf (for free). We also know some of the CrashPlan people and I believe they are good people who want the best for their customers.

Thanks for sharing that fascinating insider insight regarding non-zero-sum competition and advertising!

This whole CrashPlan Home episode deepens my skepticism about services that claim to offer an unlimited amount of a physical resource (atoms on a hard drive, in this case) for a fixed price. Code42 has mentioned certain reasons for exiting the home user market, but I wonder if an unmentioned reason is the cost of providing service to an excessive number of users who consume more hard drive space than is profitable. As a current Crash Plan Home customer, I might be one of those users, given my 2.2 TB data set. It's felt like a steal for me, but in the future, I'll likely seek an option where I can get a good deal, a good, fair price instead of a steal (which in the end results in pain and hassle and ends up costing more than what I bargained for).

Is the cost of disk space the reason why BackBlaze expunges files deleted on a backed-up computer from the backup on your servers after 30 days? Though I think you guys are cool, this policy is a deal breaker for me, as far as a Crash Plan Home alternative is concerned. At this point, I'd rather pay a reasonable price for the disk space I use rather than have an "unlimited" plan where I have to constantly look over my shoulder at my files, to make sure important files haven't been inadvertently deleted within the past month. If the cost is too much for me, I'd rather be the one that makes the decision as to which files to exclude from my online backup, rather than have the backup service do it for me.

P.S. and Disclaimer: I developed a backup companion utility (Bitrot Detector) that happens to be more relevant and useful to users of a service like Backblaze which performs file mirroring than one like CrashPlan Home which performs file versioning. However, I'd prefer it if every backup service did versioning rather than mirroring, as version-preservation is what allows you to set-and-forget a backup, rather than to set-and-constantly-worry. Though of course I want more customers for my product, I'd rather have relatively fewer if it meant I lived in a world where every backup service did versioning and fewer people experienced data loss and the resulting grief.

> Is the cost of disk space the reason why BackBlaze expunges files deleted on a backed-up computer fromthe backup on your servers after 30 days?

The original reason was to prevent customers who owned a single 1 TByte hard drive from filling it with content, backing it up to Backblaze, writing down the date, then emptying the hard drive and filling it with DIFFERENT content, backing up to Backblaze, writing down the date, repeat 50 times.

The idea was this: if the data is not important enough for you to keep a local copy, Backblaze is not going to keep it either. You aren't allowed to go down to just "one copy in Backblaze".

We chose the 30 days as something we thought of as "reasonable". For example, we expected that within 30 days you would realize your laptop was stolen so you could request a full restore. That sort of thing.

However, and now we're all furiously debating the 30 days based on the enormous amount of feedback we are getting today. We will do an analysis and if we can afford it, we will be increasing that number.

Maybe you could also furiously debate the Linux client while you're at it :)

(I can better understand the policy now as well -- keep people from using as primary offline storage.)

I'm not sure I understand the use case you mention. Is it in essence the idea of someone abusing an unlimited plan to back up more data that you'd reasonably expect them to? Can it be boiled down to Backblaze's cost of hard drive space and whether users pay enough to cover Backblaze's costs?

The use case you mention seems to be very exotic, but who knows, maybe it's not as rare as I would expect. To me, expunging deleted files because of this rare, somewhat-malicious case seems like it would cause undesirable collateral damage among your service's typical users.

Lack of file versioning and not persisting deleted files means Backblaze isn't even on my radar.

The mentions here of poor restore options doesn't sound all that positive either, but due to the former I'm not even considering going as far as trying you guys out.

The way things are looking I'm probably going to stay with Crashplan, migrating to their business plan. I'm not a data hoarder (backup set is ~400GB), but I haven't found anything else which has the same feature set for a reasonable price.

To expand on my post, the features I require are:

- Keep at least first (non-zero sized) version of file, at least one version per month (if modified), at least one version per 15 minutes for last week, last version of file.

- Ability to back up files in certain directories very frequently (ie 15 minutes intervals).

- Ability to search for and restore individual files in client, ie without sending my password to someone else.

For this I find the $10/mo Crashplan now charges to be reasonable (my current backup set is ~400GB). I would be prepared to pay more if I required more storage, though within reason.

A couple of years ago I went to use GPG for the first time in a long time. My private key didn't work. I looked at the file, and its size was 0 bytes. Cosmic ray? GPG crash? System crash while the file was being written? How it got truncated, I'll never know.

No problem, I'll just restore from backup. I have CrashPlan online backups, and local backups with Obnam, so I'll just recover it from one of them.

Every snapshot in CrashPlan and Obnam had the truncated file, going all the way back to the first snapshots. I thought I had lost my GPG key forever.

Then I remembered that I had some old CD-R/RW backups from years ago. I started going through them. Some of the discs were unreadable. Finally I found one that was readable and had the untruncated private key file.

Lesson learned: always keep your old backups. You never know which files on your system have suffered from bitrot or accidental truncation or accidental deletion--until you try to access them. It's very likely that some of them will have been destroyed more than 30 days ago.

Now keeping old backups doesn't mean keeping every snapshot, ever. CrashPlan takes 15-minute snapshots by default, so obviously I don't need every one of those going back years. But I definitely want to keep at least one snapshot for every year I've used the system, at least one for each of the last 12 months, etc.

As a personal anecdote: I was a BackBlaze customer for over a year around 2011/2012. I backed up one computer with 2 attached hard drives.

Over a period of time one of the drives failed; slowly. It would lock up sometimes; and I would restart it. Months later I realized it was losing files when it did this. And when I tried to restore those files from BackBlaze I discovered that they had been purged; and that this was considered normal behavior.

BackBlaze had lulled me into a false sense of security regarding my data by claiming to back up my files but actually mirroring a hardware failure on my local system.

I had an unhappy email exchange with BackBlaze tech support, and also Brian, and came to realize how flawed the system was.

I switched to CrashPlan at that point.

I will not consider using a product that lacks proper file versioning; and a much longer lifetime for deleted files.

Thi is a nice ego boost for you but I wish you had answered some of the questions that folks have posted.

What good is a backup tool if you a) can't restore using the native tool and b) you can't restore files from more than 30 days.

> What good is a backup tool if you a) can't restore using the native tool

I am so confused by your question? With Backblaze, you can get 100% of you data back in two ways: 1) you can get a free external USB hard drive sent to you with all your data on it, or 2) You can prepare a free ZIP file with all your data and download it. We provide a restartable native bzdownloader to help you download the recovered files.

> What good is a backup tool....

Isn't the goal to get all your data back? Backblaze does that. Maybe I'm mis-understanding your question?

> you can't restore files from more than 30 days.

With Backblaze, you can recover files for more than 30 days. You can ALWAYS get "the most recent version" even after 9 years. What you cannot do is get all the "intermediate versions" (like if you change a text document, we retain all versions for 30 days, then we only keep the most recent version forever). I do agree that would be a better product (retain infinite versions of every file). Unfortunately we would need to charge more for that, and many customers only want the most recent copy of all their documents.

I understand if Backblaze is not a good solution for you. I just want to be absolutely clear what we provide and what we do not provide.

Wait, if I delete a file you will keep the latest version forever?

I just posted a long reply to your original message, without first seeing that you'd replied to this comment (made by someone else).

If this is how the 30 day policy works, it is slightly more appealing than what I thought. However, it's still not enough peace-of-mind for me. This means that recent files are vulnerable to corruption during their first month of existence (through accidental overwriting, and some kinds of viruses and ransomware). The "most recent version" that is backed up may end up being a corrupted, useless version.

I agree that retaining infinite versions of every file would be cost-prohibitive. However, you may be able to find a healthy compromise. Maybe you could retain a ton of versions of small files and fewer versions of large files. Maybe you could provide users with a space allowance for versioning and allow them to decide which versions of which files to delete (if they reach the allowance limit) or to pay more to increase the allowance.

To summarize: You need versioning in order to be an awesome backup service (and an awesome backup service is the only thing I'll happily settle for!)

> if I delete a file will you retain it forever

No. We consider a file deletion as "the final version of the file you wish to retain". In that case you can get the deleted version for 30 days, then it is gone forever.

I'm sorry but this is a complete no-go. As much as I like Backblaze's attitude and pricing plans, this is unacceptable for a backup service. It's way too easy for files to be accidentally, unknowingly deleted, and for this to not be noticed for months or years. Any backup system that deletes missing files is not a backup but a mirror.

Without some form of versioning, there's always the risk that the only backed-up copy of a file is a corrupted, useless version. This is the best summary of everything else I've said here.

Indeed, I nearly lost my GPG key due to undetected corruption, because the corrupted file had been backed up for a long time. I only recovered it from a very old CD-R backup I made years earlier.

Keeping old snapshots (e.g. one or two per year) is an absolute must.

Do consider a (paid?) add-on similar to Dropbox's Extended Version History [1]. It'll at least protect against ransomware and other mishaps.

It'll also be nice if you could provide an option to get alerts if a certain subset of files change. I have folders on my NAS (I know Backblaze doesn't back up servers) which are basically files from my old desktops that I'll sort through some day in the future, and which are highly unlikely to change, maybe ever.

[1] https://www.dropbox.com/help/security/extended-version-histo...

Reply on news.ycombinator.com