Hacker News Reader

Comment by flohofwoe | Hacker News Reader

Git keeps the entire history on your local machine, this becomes a problem if your project grows to several hundred GB (not untypical in game dev). Even SVN was much better for working with large repositories, you only needed a big server. Git is quite nice for "source code only projects though".

View on HN

This is true, but it is technically an implementation issue. Git supports shallow and narrow clones. The data model works fine with only partial local knowledge. That being said the "check out all files to disk" approach does have fundamental limits for many operations. It is one of the very few operations that are linear with the tree size but it isn't fundamental to the abstract model of Git. To solve this something like a virtual filesystem that fetches new data as required and tracks changed files can make almost all daily operations scale with the size of the patch, not the size of the tree. The data model for this tracked change would be basically equivalent Git's index structure, just applied to the working tree instead of leaving the working tree on a general purpose filesystem.

That being said the unfortunate truth is that many, many, many users of Git rely on the implementation as the API, directly reading and writing to all corners of the `.git` directory. So even if the core command could support something like this (or a wrapper could be provided) it would likely run into a lot of tooling issues.

Microsoft created the git virtual file system to address this so that they could move Windows to git. It has since been replaced by “Scalar”.

https://learn.microsoft.com/en-us/previous-versions/azure/de... https://github.com/microsoft/scalar

I have a lot of huge files and use SVN with it. It's dirt cheap to host, next to no memory requirements, and it comes with a web UI to boot because it's built in top of WebDAV. You can literally open a repo in the browser.

The only downside is the per-file versioning, the update needs to be run on the top directory otherwise interesting problems happen.

git supports shallow clones.

Does Git LFS help with this problem?

Have seen various teams have major issues with LFS - becoming quite a support overhead to keep art teams productive.

Yeah exactly this. Git LFS is supposed to fix the problem, but I haven't seen it working flawlessly either yet.

Interesting, I had issues using Git LFS to store images for my personal site. I thought I just didn't know how to use it properly...

Not Git LFS, but Git VFS, now replaced by “Scalar”:

https://github.com/microsoft/scalar

Overall LFS feels a very hacky tacked on solution with warts.

Git LFS is horrible if you mistakenly add some file(s) to LFS.

Restoring to LFS-less state is supposed to be easy but it is always very painful.

There should be a way to tell repo - I do not want any LFS at all. In practice this is painful, and it is easier to start a new repo....

Also for some reason Github/Microsoft decided to "monetize" LFS. The freebie limits are 1GB for hosting AND transfer! Then the charges get very expensive.

I am not sure why Github LFS is priced like some AWS S3 plan not OneDrive plan.

GitHub charges for LFS and has since they initially introduced it, way before Microsoft was involved. Also, the pricing model is changing and will include a minimum of 10GiB for free (250GiB for Team/Enterprise customers). It’s cost-recovery and abuse-prevention, though, not some nefarious scheme. And funny you mention AWS S3... what costs might GitHub be recovering? ;)

Microsoft’s homegrown LFS implementation (in Azure DevOps) does not and has never charged for LFS. It’s nice to have friends with a cloud blob storage service!

Source: I was the product manager for Azure Repos’s LFS server and am currently the product manager for all things Git at GitHub.

Reply on news.ycombinator.com