Recognizing the limitations of cloud drives
backblaze.com
backblaze.com
One day a ticket was sent my way. A customer complained that some files were missing. Once I got logs from that customer's computer; it was clear, someone, or something, deleted those files... Four months ago.
A recent cost-savings initiative bumped the deleted file retention to ~3 months. The retention window had come and gone, and we deleted the customers' files from their recycle bin.
My response was basically: We need a longer retention plan if we're going to call ourselves "backup." The point of "backup" is to restore in cases like this, where someone (or something) accidentally deleted files.
I even had that lesson with some important music files. I shared them with the person who originally recorded the files, and they dragged and dropped them out of the shared folder. It wasn't until a few years later that I realized I needed to restore them. Because I had the VIP flag, I was able to restore them.
I'm using local backup software and it's pretty clear to me that versioned and trashed files still occupies space until I delete them permanently. If you provide 1TB of storage space, make it clear that the trash and versioning occupies some of that space instead of a random retention period. Ditto if it's unlimited and priced by usage.
Find a sweet spot where you can satisfy most clients at a manageable cost. Offer problem clients a higher data retention for a premium.
I'm general, I think this is a good approach.
The consequence is that there is a certain class of user that won't understand this and will get confused when their account hits quota, but they don't see those files on their computer.
Another consequence is that we don't sync recycle bins, so it wouldn't be straightforward for users to purge it.
I personally think that moving deleted files to something like Glacier, (slow and cheap, think tape in the cloud,) and then having a separate recycle bin quota, is the best approach. (https://aws.amazon.com/pm/s3-glacier/?trk=75fef084-bf49-48b4...)
I pay for one year with Backblaze (on the cheaper plan, it’s much shorter but they’re explicit about it) and I expect one year and they hold themselves to that one year. It’s a number we both decided on ahead of time.
For something like Google Drive or Dropbox, I expect no retention at all tbh. It’s a plus if I need it and they have it.
That's confusing for non-technical people, and it makes the product more complicated.
Corner cases were the source of (almost) all of our bugs. (A configurable retention window will add corner cases.) What happens is that one customer will have a particular combination of corner cases that no one else has, and then that customer hits a bug that no one else will hit.
What I should also add is that bugs that only effect a few customers will be very low priority.
IE: This is why software with lots of options is often buggy: Options create corner cases that aren't high priority to debug.
This comes in handy when my local Time Machine go as far as just about 6 months or so. I might need to upgrade to a NAS and hook up a larger capacity drive.
Will also need something to control software that goes really wild with new versions, or some specialized rule sets for software repos, video editing, etc.
Ransomware & compliance escrow would also work great for this, you could have contracts that wont allow deletion after X months no matter what the other organization says to help prevent social engineering or account take over attacks.
They do make a good point about full computer backups though. Id like to see support for automatic Time Machine ingestion by these clouds
TLDR: ¯\_(ツ)_/¯
"Virtual" and "air gap" is quite frankly an oxymoron. The two concepts are just at odds with one another. This article gives me misgivings about Backblaze as well.
* run on an immutable operating system,
* enforce strict (local) whitelist-based firewall rules,
* pull, not push, data from other servers,
* restrict remote administration to a physically isolated network,
* or disable remote administration entirely.
I feel like that's as close as you can get to the benefits of air gapped environments without the drawbacks of servers directly exposed to the network. Or, well, unless there happens to be a 0-day in the TCP/IP stack. It depends on the threat model I guess.
I know this because I am meticulous about backups and these particular files are encrypted private records. I don't trust software but I especially don't trust that backups will correctly handle sparse disk images since they are spread out over hundreds of files; so I always keep the output of mtree in that directory for the current version. I had checked the mtree file, edited some of the files contained therein, and regenerated it the night before; SOME but not all of the sparseimage subfiles had rolled back to October. Changed timestamps, changed hashes, the image did mount but the enclosed files were from October (they have date-stamped rows).
So when I went to open the disk image, the next day, and ran mtree to test the directory first, it noted the changes above.
I moved to icloud drive about two years ago. I'd previously had some problems with Apple's sync blowing away some notes in notes.app so when I moved everything from Dropbox (where my paid account for 7 years never lost or corrupted a file to my knowledge - and my knowledge is good because this mtree habit I just described goes back a decade!) to iCloud, I kept this habit up, and good thing.
To this day I don't know how it decided October or ended up replacing them. Maybe there was some local cache, I don't know. I did not revert the file in iCloud and nobody else has access to my account. Further, it only partially rolled the image back.
I had never had a problem up to that point, but I will say, with storage, once is too many times. Back to dropbox.
Apples cloud is drippy.
My backup discipline is excellent because back in the early 2000s we lost a drive which I had not been backing up and it had months of contract work on it; I ended up paying $2500 plus $30 per _CDROM_ burned for the content from that drive... it was just barely worth it.
In other words, iCloud is one “thing” even if it claims itself to have multiple versions/copies/backups.
But the scary thing about this is that it might have gone undetected- silently reverting a file and it not being noticed is death-tier horrifying.
* Enough storage at a reasonable price such that I "don't have to think about it"
* Some kind of online-aware virtual file system such that I can see my "online only" files right next to my "offline" files, and only download the online ones if I explictly download or try to access them.
I find many "hardcore" techy backup solutions recommended on HN and elsehwere miss this second bit. I don't particularly want to manage uploading a perfect byte-for-byte replica of my filesystem every month or whatever. I just want it to be magic. Perhaps occasionally I'll put things on my own external SSD. I guess if Microsoft data centers get nuked or whatever I'm SOL on the last 3 months of data updates. I don't really care. I just want the cloud to function as an extension of my local hard drive in the most seamless way possible.
Use rsync.
Sadly for me only iCloud offers this with their 50GB for $1 tier. I basically need cloud sync to be a USB stick that’s plugged into all my computers simultaneously.
Self hosted is not for everyone but at what point is your stuff important enough for you to really take responsibility for it. Do you simply Dropbox or OneDrive or whatever or do you do it yourself and self host or get a family member/friend do it or engage someone to do it for you.
You do have choices. Even if you lack the skills to self host you can engage someone to set it up for you. You probably don't plumb your own bog 'n' bath and yet you own a toilet and bath/shower/whatever that works. A trade has fixed that up for you in return for some consideration.
So, do the calculus for yourself: How important is your data, what sort of losses if any are sustainable. What are the risks ... have you done a home fire risk assessment - do one, its not hard. Fire blanket in the kitchen (ban towels drying on the oven), one shot ladders (that sit in the bottom of a wardrobe - inform guests - make a joke of it) for upstairs bedrooms, foam fire extinguishers - one or more per floor. Plan for escape. £500 or so per five years is a very rough budget for hardware. By law you probably should also have battery backed, mains powered smoke detectors already, so I don't count that in the budget. If not, then get some and test them.
UPS ... get one, especially if your storage does not include a battery backed RAID controller. File systems are rubbish with holes in them. APC and others do several small units for home use with domestic plugs (in the UK we get our fearsome three pin plug sockets). Get PoE+ switches - Netgear and others do reasonably cheap switches. That smart doorbell really should be PoE powered with UPS backup ...
If you are reading HN then there is a good chance you are either an engineer of some sort or the sort of person who thinks like one. Once you have worked out the risks and costs and yes, some of those costs might include being kicked from here to tomorrow by a partner (where are my photos?), then you can create a plan. I'll leave it up to you how you squirrel away the costs but please ... pretty please ... do a proper plan for this sort of stuff and whilst you are at it do a reasonably wide risk assessment. You might want to cover "everything" as you might at work - it can be quite cathartic.
BTW Syncthing is not exactly a backup solution; it's a sync solution, with an ability to have staggered versions of changed files.
Something like Restic or Kopia is a proper backup solution, with encrypted versioned data good for off-premises storage, and an ability to mount a point-in-time backup.
Ironically, I have a snag with my personal photos syncing from my phone which will probably involve some SQL but I did try to go off piste in a way that your average user doesn't.
NC is very reliable and since a few years ago, the phone clients have stabilized nicely. The desktop clients have been pretty decent for much longer.
Across the various systems I manage I think I've seen most or at least several corner cases. My home instance has few users with a large number of files. My wife's photo collection is legendary. The instance with the safety docs involves a lot (lot) of PDFs generated by a few people on PCs, that are shared with many tablet/phone based users over variably connected links. If the docs are not there, some things are not allowed to happen.
I've got some instances behind HA Proxy and some behind whatever MS's Azure proxy offering is called. One is behind Apache only and another nginx only but they both do have quite a lot of log file watcher (OK call it an application firewall) and network traffic analysers.
Syncthing has its place but I need rather more in most of these cases and NC has a lot of modules. For example the safety useful system has rather a lot of logging for doc creation, amendment and access. My life becomes briefly rather miserable if my wife's photos are not available for posting on FB. NC has an "impersonate user" thing which cuts through quite a lot of the IT to end user confusion. You can look at their account as they see it. I also have Mesh Central ...
A backup is a binary equivalent of all the source data. True backups cannot be modified, and exist as a point-in-time reference snapshot.
Yes, all backups have retention periods. That doesn't make them not backups.
> True backups cannot be modified, and exist as a point-in-time reference snapshot.
OneDrive does keep point-in-time version history and allow you to access and restore, but not to modify, those point-in-time reference snapshots.
SpiderOak ONE doesn't:
https://spideroak.support/hc/en-us/articles/115001933406-The...
They keep the deleted files forever.
I've used them for over a decade now. Have certainly restored files deleted over a year prior.
> Since the files in the Deleted Items bin are still retained in storage, they count towards your storage total. To permanently remove items in the Deleted Items bin from your account, select the files and push Remove.
The default behaviour in MacOS for Trash is no automatic deletes, so I guess you get this forever backup so long as you don't turn on automatic deletes for Trash.
Timemachine also keeps backups until you start running out of diskspace.
Neither service deletes files from your backups.
If you add sync to a folder that was originally created under a different account, it will use stubs by default and download on-demand (though you can override this). But the redundant storage of those files is not your problem, that's between OD and the other account.
It’s just a trade off based on how much time and comfort you have as a personal user.
Unless you live in a Backblaze data center, using Backblaze has always implied that your data is stored in at least two separate locations.
- super slow; scrolling often leaves you in a blank space for seconds when it's catching up; elements showing up on the page after you navigate there pushing things away from or under your cursor
- breadcrumbs in the folders sometimes showing, sometimes not
- different directory tree browsers in different parts of the app
- folder settings split between multiple different places. For something you have to go to Admin console, for something you have to go back to normal viewer
- multiple types of flash messages, showing after different actions, often doubling themselves
- actions in dropdown menus for the same thing ordered differently based on where you currently are
I'm so happy that we are migrating away to our own solution
Our only choice is the lord and savior, Onedrive, since our BIG_COMPANY is all-in on Microsoft. S3 is allowed as well but that's mainly because it's used in so much other than file sharing.
Dropbox is the devil. Google drive is virus central. Backblaze, Box.com, and the rest aren't even on the radar, but blocked by category.
> Additionally, it’s essential to note that cloud drives, which are primarily sync services, do not offer comprehensive data protection.
There is no cloud. It's just other people's computers, and they don't care about your data except the parts useful to them.
Time to fire up the old RAID.
3-2-1 is a backup strategy.
Example: A small company with data they find important but it isn't a great volume and they don't have a huge budget. They do a backup of everything once every week or month (it doesn't matter).
Should a drive fail, the backups could be handy, but who wants to lose days of work? And so people turn to RAID.
Note that as your strategy/need evolves toward continual backups, backup and RAID begin to merge, expensively.
Pay once, store forever.
I'd say an even better comparison is glacier deep archive. Let's say an average of one retrieval per decade. $2.18 per month for storage, $210 per retrieval (which is 90% egress cost that you might be able to avoid), 600 years at current pricing.