Hacker News Reader

Comment by rl3 | Hacker News Reader

Although this is large for a company that deals mostly in web-based projects, it's nothing compared to repository sizes in game development.

Usually game assets are in one repository (including compiled binaries) and code in another. The repository containing the game itself can grow to hundreds of gigabytes in size due to tracking revision history on art assets (models, movies, textures, animation data, etc).

I wouldn't doubt there's some larger commercial game projects that have repository sizes exceeding 1TB.

View on HN

I am working on a game where head is 1TB. On top of code and assets this size includes a few full builds and a full set of game-ready data (the data build process takes something ridiculous like 7 hours, so that's done on a server and it checks the result in). All in the same repository.

1TB is rather a lot. My previous record was 300GB and even that seemed a bit much. But it is very convenient having everything in one place, including all the stuff that you only need occasionally but is handy to have to hand, such as old builds of the game that you can just run in situ and installers for all the tools.

(I don't know what the entire repository size must be like, but many of the larger files have a limited history depth, so it's probably less than 5-10TB. So not exactly unimaginable, though I'm sure buying that much server-grade disk space - doubtless in duplicate - might cost more than I think.)

Interesting. Where do you work if I may ask?

WOW. BTW what game is that you are developing ?

Half-Life 3 I assume.

But they surely don't use git for that, right? In scenarios like this a versioning system that does not track all history locally would be a better fit.

Perforce tends to be big in game dev because it does better with repos full of giant blobs, has locking for working with them etc.

Perforce is king in game development. It's also the only place I personally still use Subversion.

I think most use Perforce.

As the other replies say, Perforce is dominant in commercial game development.

However, Perforce does have Git integration now, allowing for either a centralized or distributed version control model. Considering the popularity of Git, I wouldn't doubt smaller Perforce-based game projects are going the DVCS route.

Also, hypothetically speaking, consider if you had a game project that would eventually grow to 1-2TB in repository size. If you spent $100 per developer to augment each of their workstations with a dedicated 3TB hard drive, you would have an awesome level of redundancy using DVCS (plus all the other advantages). I know it's no replacement for cold, off-site backups, but it would still be nice.

It could use git-annex[1]?

[1] https://git-annex.branchable.com/

There are a lot of non developers using source control for game development. Perforce makes it easier for graphic designers and the like to use the VCS and not have to deal with a CLI. Perforce just handles binary blobs well.

Strange as it sounds, git-annex doesn't really do file versioning very well.

Are you talking about git annex assistant or git annex? git annex does file versioning very nicely then again it doesn't work on Windows so that's probably not very useful for most game developers.

git-annex does file versioning, but it's extremely uncomfortable to use (and I say this as somebody totally comfortable with git) and I'd never expect an artist or other only-semi-technical person to use it even if it worked with Windows. Especially when Subversion or Perforce are right there.

Git does have support for "shallow" clones, that don't pull down the full history but only the recent history.

shallow clones the only issue for games. Another problem is, unlike code, art assets usually can't be merged. You can't merge 2 photoshop texture files or 2 edits to a 3d character made in Maya.

So, you need some central system to manage the assets so that people know "hey, don't edit this file right now because so-and-so is editing it".

Ideally you'd like to know this BEFORE you start editing. In other words, you don't want to spent 15-60 minutes editing something and only on saving or trying to check in get told "hey, sorry but someone else was editing this file so you'll have to discard your changes and start over". Some editors are better at this than others.

You could try to write something on outside of git to help with this but why when P4 already provides this?

Even better would be collaborative real-time editing.

> You could try to write something on outside of git to help with this but why when P4 already provides this?

Maybe because P4 is kind of a PITA? I used it for 10 months on a project (without any noticeable art assets, even; this was just code) and it regularly caused problems. The company had someone whose sole job was to administrate P4, and it was sorely needed.

Of course, it's been many years, and I no longer remember details about the precise problems encountered, just the overall frustration. Although the one thing I do remember is the aggravation caused when a coworker accidentally locked a file they weren't editing and then left for the day.

I wonder what the biggest code base in the world is.

Like you say, games include all sorts of binary assets. Any idea how much actual code is there?

The Linux kernel is only 175MB https://git.wiki.kernel.org/index.php/GitBenchmarks#Estimate...

The F22 has some 1.7 million LOC http://en.wikipedia.org/wiki/Lockheed_Martin_F-22_Raptor#cit...

This graph shows some pretty big things http://dailyinfographic.com/wp-content/uploads/2013/10/1276_...

looking at the plot, i wonder what the heck is up with healthcare.gov... what on earth does the site do that really requires 5e8 loc?

Support Internet Explorer

Absolutely! BF4 with the expansion packs is > 40GB as a compiled binary and compressed assets. The source (with no history) must be gigantic.

That is mostly data and assets though.

git can not really deal with binary files well, if you have lots of png/gif/mov/mp3/jpg/etc to track git is not the option for you I think. git excels at source code, in txt format that is.

Reply on news.ycombinator.com