Hacker News Reader

Comment by tails4e | Hacker News Reader

The file size limit of git is a bit annoying. I've moved to svn for a few projects that required large binaries to be shared and version controlled along with their source (for hardware design projects) and hit the 100Mbyte (IIRC) file limit. Im sure it could be fixed, but it is a limitation that has prevented me from using git in some cases

View on HN

Git lfs is what you want.

The problem with this software is that it uses a different channel to transfer the large binaries.

If you're using git-shell or git-daemon to host your git repositories on a server, then you can't use that alone to also manage transfer/access of stuff that is managed through git lfs.

Broken on Windows.

Why isn't is integrated into Git out of the box? Why do I as a user have to be aware of a separate large file support? Why do I have to install and use a separate tool for this?

Everyone immediately says that when this topic comes up, but afaik it doesn’t even transfer or store files as deltas. Each time you change 2 bytes in a multiple GB file, you get to transfer and store an entire duplicate. IMO that is not a genuine solution to large files.

In fairness, that’s because gitlfs’s primary purpose is to store binaries where changes in the representation don’t actually map to anything useful. There definitely is a separate use case out there for storing large files where only deliberate changes are made to the objects.

Transferring only deltas is a performance optimization, not a question of semantics. Git-lfs needs to be able to perform binary diffs to save you from uploading 2 GB every time you touch a 2 GB file, not so that it can display useful diffs.

The tech to detect binary diffs has existed for a very long time as well. Since the previous revision is present on disk, there isn’t any good reason why they can’t at least delta compress the network transfer and validate hash on the other end.

But beyond that, what’s keeping the server from storing these deltas instead of full versions? Only argument I can think of is that server side corruption of a single file would break the chain of deltas and you’d lose every child.

But surely that can be solved using some sort of redundancy.

I agree; cvs, svn and git work ok and don't need that much change, what needs improvement is separation between code and media!

I'm making a simple .zip over HTTP solution for my MMO now.

It just has a naming scheme with dates and you send your last .zip date to the server and the server sends a list for you to download.

The good part with dates is that you can consolidate the data retroactively without breaking the system!

Put them in an artefact repository and pull them down at build time?

Reply on news.ycombinator.com