Hacker News Reader

Comment by WoodenChair | Hacker News Reader

The author suggests using git. But be warned git doesn’t work well with iCloud Drive. I used to have hundreds of git repositories in my iCloud Drive. Periodically, doing a big commit would trigger my whole multi-hundred gigabyte iCloud Drive to resync with Apple’s servers. This would take a day to complete. I went through Apple support up to quite a high level to try to resolve it and they never did. Now I keep my git repositories outside of iCloud Drive. The trigger seems to be if you ever change hundreds of files at the same time in quick succession before the last set of big changes finished syncing.

View on HN

I thought it was common knowledge that you shouldn't mix git (and similar VCS) with cloud drives and their cloud based version control (iCloud, OneDrive, Google Drive etc.). The two don't mix well and there's room for all sorts of weird and unexpected behaviours especially if you have multiple devices synced to the same cloud drive.

I'm using syncthing to keep bare repos of my git dotfiles and alikes. Never any issues.

I do the same.

I think the golden rule here is to only work directly with the repo on a single device. Otherwise there is a lot of useless churn and more potential for conflicts.

If you want to work with it on multiple devices, you can still sync it around, just work with a new clone of that repo rather than modifying it directly from multiple places. Then push your changes into the synced repo and act like it's really remote instead of local.

This does use more storage, however.

Yup, you definitely shouldn't. I don't know if it's common knowledge so much as some git operation will start failing after the fifth commit, so you figure it out pretty quick.

I do have to say, I've never understood why git doesn't work well with cloud drives. File management seems like a straightforward enough set of operations that ought to be bulletproof. I've never understood what precise operation gets corrupted when using git, and how that's possible at all.

And I don't know if the fault is the cloud or git. Is it that the cloud returns errors 0.5% of the time, and git silently ignores those instead of retrying, and corrupts data? Or is it something about git reading and writing so many files so quickly that operations on the cloud drive somehow get lost or out of order but without generating errors? Both seem equally implausible, and yet...

I think it’s because git provides atomicy, requires it, requires you to make decisions on merge conflicts, etc while cloud sync makes invisible arbitrary decision of which file is newer, if there is a “conflict” like the same filename but different hashes and sizes, it will happily “clobber” your git database records by arbitrarily choosing which file is “better”

Hi, original author here. Thanks for sharing.

My problem is, that I need to use iCloud Drive for some important files because the app I need to use doesn't support another way to sync.

Without git I wouldn't have figured out which files have changed. (And even with git, I can't be sure that I caught everything. I guess git content can also be altered by iCloud sync conflicts…)

you might store the git repo on a Dropbox folder (not a good idea in general, afaik clouds aren't reliable enough for git, but as a backup), this way git content can't be corrupted by iCloud?

Or maybe even locally if you don't need to commit on other computers, then you'll be certain that git is correct without any cloud intervention

Can I store the hidden git folder outside of the repo content? Not sure how to say.

My problem is, that for the time being, some of my files needs to be on iCloud Drive because I rely on one app to edit/create them. Everything else I moved out of iCloud Drive.

Yes the (in git terminology) repository (.git folder) can be completely separate from the working directory (folder on your iCloud drive)

Thank you! I didn't know. I'll research and move the folder.

Git doesn't even work well with large binary objects.

Yes, and git doesn't play video files either, because it is a version control tool for source code, not random binary data.

It's a different topic but often there's binary data that evolves along with source code, so it makes sense to put it in the same repo. Also, other version control systems make no assumptions about the underlying data, except when diffing and merging. People want and need this, and it seems reasonable.

> other version control systems make no assumptions about the underlying data

I most definitely used Git with binary data without any issues. It makes no assumptions as far as I can tell. It's just not great with very large binaries.

I think you cannot make a completely generic indifferent system in finite time. Maybe your system is optimized to efficiently work with 5 1TB files, or 1 million 5KB files, or just nicely handles a reasonable amount of text files in a neat way while being not extremely complex or hard to maintain & deploy.

If Git does version control better than other control systems but only does it for text then I might be fine with that, I think it's reasonable to have a scope.

My usual approach with valuable iCloud Drive folders is to have an automatic process rsync them somewhere else and auto-version them there (Git, Dropbox, etc.). This wouldn't be great for very large data, but most of what I care to aggressively version and keep that lives in iCloud Drive is text or otherwise small, for which having two copies is trivial.

For what it's worth, git also can fail in some pretty bizarre ways in Dropbox for much the same reasons. I've used git-remote-dropbox [1] in the past to use Dropbox as a git server.

[1] https://github.com/anishathalye/git-remote-dropbox

I'm truly curious why would you use iCloud to back up git. Why not just push your git to the remote?

You can make a local git repo without having a remote server. Then they’re backing it up off site with iCloud.

Sure I understand. However it's free/inexpensive to use a service like GitHub/GitLab/Stash to save your work remotely.

Hi, original author here. In my case it's not that I want to use git. I just need a way to see which files got changed. At least until I can move the last ones out. I still rely on one app that only syncs via iCloud Drive.

The reason I did use them together for a few years is when I bought a new computer or went between two computers under the same iCloud Drive account I could pickup exactly where I left off in my development work without even thinking about it. Obviously, due to the issues it's not worth it though.

https://stackoverflow.com/questions/35853139/can-git-and-icl...

Reply on news.ycombinator.com