Source Control for Art Assets Must Exist
hacksoflife.blogspot.com
hacksoflife.blogspot.com
- Working with large files
- Getting only the needed files form the sever, not all
- Not wasting space with .svn/.git copies of the files
- Reasonably fast.
They have quite some customers in the video game industry because of their support for huge projects and large files. We are using it for a different kind of application with similar requirements (100000s files, size up to GBs range per file). Perforce is well suited for the job and has good 24/7 support.
GitHub has just become so ubiquitous, you start to forget other source control systems are out there. If I wasn't still using SVN at work, I'd be in git all the time.
I did not try this though.
Which aren't bad, but may not be what you're looking for.
I personally host a perforce server on google compute engine for around $15 a month (g1-small + whatever size persistent disk you want). Relatively easy to set up (they provide repositories for ubuntu and fedora iirc), configuration is largely interactive when setting up or through the P4Admin tool (which allows you to very easily setup permissions, depots, etc).
I know that isn't fully managed like github and doesn't provide issue tracking, wikis, etc, but I've found it fairly easy to deal with if you know a bit of how to setup a linux VPS.
I haven't had a chance to use git-fusion "in anger" yet, but I hear lots of good things for treating a p4 like a git endpoint to allow devs to work in a DVCS manner while keeping all the good parts of Perforce.
Perforce for source control. A FUSE file system. You configure it to point at a revision (or at head) of a branch (almost everyone is always on the main branch). Then it makes a drive (or folder) look like the Perforce repository, but read-only. It caches files as you read them. Then, if you try to make, modify, or delete an asset, it stores those locally, but makes it appear to be in the same directory as the rest of the repository.
I think this would be perfect for art assets.
More (potentially stale) info here: https://lwn.net/Articles/380983/
https://github.com/bup/bup/blob/master/README.md#things-that...
Because of the nature of bup's solution, there are some operations that haven't been implemented because they would be prohibitively expensive. Pruning old files for example -- the equivalent of git's "filter-branch" just doesn't work.
(I am a programmer, and I still find Git to be nightmarish. Sadly our company adopted it because it's trendy, not because they did any actual research or study about our source control needs-- the lack of centralized file locking bites us in the ass every day.)
The reason they stuck with SVN that most interests me is that SVN has some GUI clients that don't completely suck. Sadly, not true of Git. At least not on Windows.
No thought or organization, and certainly no usability study, went into SourceTree.
I would never argue that it is user friendly though.
Visualising changes, multi-checkouts and merging is impossible.
Seems ripe for disruption? Or will this stay an unsolved problem forever?
I think this is a big mistake programmers make when trying to create VCS for other industries. A lot other fields with binary files don't care about merge as much. Like electronic design automation: track every part's history and even every module on a board or die layout. But, practically it's difficult to have two people productivly work on the same small piece simultaneously.
[edit] I also think a bunch of the reasons programmers require merge are because code is organized into files. Organization into files is arbitrary and often doesn't correlate particularly well with logical structure.
We pivoted. The main issue we faced is the way the electrical engineers work and how difficulty it is to change this. It was incredible hard the explain these people how to properly use a VCS. We focused on a simple interface and tried to optimize on the way electrical engineers work. No chance!
Obviously the same applied to LayerVault [1].
[1] http://thenextweb.com/apps/2015/03/11/layervault-the-version...
Sadly I don't think diffing & merging of art assets will ever become viable and locking can only work if your assets are controlled at an OS level. The simplest solution is to hide a file if it's locked.
Perforce is not good at dealing with large assets one artist uploading 1 video file can grind 300 other people to a halt. I know this from bitter experience. You can solve it to a degree using standing servers - but it's tricky and expensive to get right.
Fuse is interesting but doesn't work with Windows. WebDAV is old and cranky and half broken. Anything that saves immediately will lock your editing software up while the file travels over the network (and with art files that can be minutes! (think video)).
Dropbox is the strongest contender here as they allow a quick save followed by a slow sync. Sparkleshare has the right idea and working Windows client (it uses Git behind the scenes so isn't really suitable for video assets). Camlistore is an exciting technology based on the Plan9 file system (and Git) it should cope with large files in theory though I haven't stress tested it. It's still young and there isn't a good visual client yet.
I'm coming at this problem from the world of D.A.M. (yuck) but worked for years in the AAA Games industry (many, many, giant art assets). Traditional DAMs have never been any good at dealing with assets still in development. Hopefully we can change that, however there are so many problems to overcome (see above) that its unsurprising no-one has managed it yet.
Drop me a line if you have any useful/interesting use cases.
You also avoid the large asset problem by always keeping the large assets in the "cloud".
disclaimer: employee of the maker of clara.io
Storing large files somewhere outside the working copy is only part of the solution. Working with artists is going to be a bit painful unless you have centralized file locking, or some other self-managing means of preventing two people modifying the same file by accident. In theory, you could use merge tools for the binary files that artists manipulate, but in practice, these tools don't seem to exist.
(I can't speak for hg - but an issue with git is that you really have to have a good mental model of how it works to use it effectively. This is something programmers like - or, failing that, are at least practised at doing. But it's really not the sort of thing artists seem to generally enjoy. Certainly very few of those I've worked with...)
I am convinced that file locking is not the ultimate solution, even for artwork. Someone could decide to change the colours of some character while someone else could decide to redraw her arms. Both changes make sense together.
What we need is specialised diff and merge tools for artwork, which nearly any VCS already allows you to plug in. Even github has a WUI for diffing images. File locking is a workaround with other well-known problems. Merging divergent work is not any more annoying for artwork than it is for code.
Pixelapse seems to be presenting a compelling case on what VCS for design could look like:
> I can't speak for hg - but an issue with git is that you really have to have a good mental model of how it works to use it effectively.
This is widely touted as a huge difference between hg and git. You don't have to understand hg's revlog format to use hg, but you need to have a solid understanding of blobs, trees, commits, refs, and hashes to understand git.
What you do need to understand for both is that someone can have modified the same file that you're working on while you're working on it, and it is OK that they modified your file at the same time...
The thing is there's a good chance the people you're working with aren't very technically inclined(which is why they're such awesome artists, they have other focuses). Anything more than the simplest scheme is going to fall over at production scale.
It's been tried many times, the tools that you are talking about(Maya, Photoshop, ZBrush, etc) don't even consider merging with their large binary workflows. The closest you get to this is referenced scenes in Maya and that requires a good technical artist to set everything up and police things so that people don't touch things they don't understand.
Then we need a simpler scheme that still allows people to understand that two can work at the same time. Everyone hates merging regardless of their technical ability, even software developers, but I refuse to believe that it's a hopeless task to build a tool for merging artwork.
You're welcome to take a crack at it, however you're looking at hundreds of hours of work to reverse-engineer all the formats that Adobe and Autodesk use plus whatever new tools are on the horizon that you don't know about yet.
AFAIK there isn't a single tool out there that can render a PSD to PNG reliably(aside from scripted photoshop) so thinking we would be able to cover the entire suite of tools is a bit naive.