Ideally it would be possible to build all assets (maps, models, textures, binaries) in a game from source files to the format that's distributed to people.
Unfortunately due to the build times for certain things like maps it's not feasible to rebuild everything before every release. Combined with the fact that the files are relatively large means that we can't store everything in one git repository.
I'd be very grateful if anyone knows a solution to this.
We are. 2 man team and have our own hackeneyed way, some in house bunch of Python scripts to build our game assets. As we do this seriously as a hobby it works :). But now we are at a stage where the 2 games are coming along well and to go full hog, I would choose plastic.
It has integration with Unity and gutsync as well so migration is easy and with 7$ pm cloud plans for 5gb assets works for us.
Last when I had checked plastic it was out of reach out hobbyist / indie game devs. Hope this helps.
GitLab have it in the Community and Commercial Editions: https://about.gitlab.com/2015/11/23/announcing-git-lfs-suppo...
As for the git problem, why not use svn or perforce for the game assets? They handle binaries much better than git does.
Using svn for the game assets would work but to build them we would still need the shaders for example, which would live in the git repository alongside the rest of the code of the game. The resulting problem is that the asset git/svn should be triggered to rebuild when the code git changes. I'm looking for a tool that could help with that.
Thanks for the suggestion of perforce, I'll look at that.
Lucky you. :)
This isn't a simple problem and I don't know of any one size fits all solution for this. From what I could tell, every company comes up with its own asset pipeline that fits with their art, code, test and release processes.
I came up with a half assed solution myself back when I was still working in games. We had a separate build server just for assets. It ran a script every hour that pulled down both the code and art repos and watched for changes in certain folders. The shaders, 3d models and textures were linked via naming conventions. Through those naming conventions, the system always knew where to find all of the relevant pieces of any given individual 3d asset (in theory, anyway). It would then create the assets and check them into a separate SVN repo. This system worked ok, but it had a number of problems.
One problem was the naming conventions. The artists didn't much pay attention to file names, so any new asset they created would inevitably not get built because it didn't adhere to the conventions. We thought about replacing the naming conventions with a json file describing every 3d asset, where their shaders were, where their 3d models were, any special settings, where they should be saved, etc. but that would have required either a GUI application to help manage it all or a coder whose job was to manage the data files. The former would have taken time and the latter wasn't much better than what we were doing already. We still needed a coder whose job it was to fix asset build issues, which would pop up on a regular basis.
Another issue we had was build revisions. Game builds are complicated enough and we added the additional complexity of having to specify a branch name and version number for the assets used. It doesn't seem like it would be an issue, except when you have parallel art work going on in parallel branches along with parallel dev work going on in parallel branches things break. In retrospect, we should have put our foot down and said there is only one official asset branch and all dev builds will use the latest version of that branch, but we didn't.
Sorry for the essay, but I hope it was of some use to you. If you ever come up with a universal asset pipeline, sell that sucker! :)
More generically, I'd suggest you check out Mercurial with the Largefiles extension.