So basically rsync?
Honestly synchronizing 50GB of contents across the globe doesn't sound that terribly novel or difficult to me, or the article makes a pretty bad job of explaining the difficulty.
Honestly synchronizing 50GB of contents across the globe doesn't sound that terribly novel or difficult to me, or the article makes a pretty bad job of explaining the difficulty.
50GB is not the total size of the files needing to be transferred - some of the files were 50GB. Knowing the total size would have been interesting for the sake of discussion.
But at any rate the size of the individual files is not what matters. The important part is the total size of the game files and the average change rate. I would be surprised if they weren't able to synchronize everything with a good internet connection and a bunch of rsync scripts.
We use a lot of rsync.
The article title is misleading. They are actually talking about the internal workflow.
Raw assets, art assets, and such, can be pretty large.
When you have a studio in Country A doing PC, a studio in country B doing ps3, all wanting the base raw data but (presumably) transforming it in different ways, between how the build actually happens or artists tweaking something for a platform, it'd become a nightmare pretty quickly if you were to send it across a company private network (often just VPN pipes across the internet itself).
The article is, at it's heart, an ad for a company called 'Panzura', which sounds like an enterprise-specific dropbox. Boxes 'on-premise', security guarantees, and probably pretty expensive.
Most games' source data is much larger than the shipped data size for a number of reasons, mostly because files designed for human editing encode a lot of information not needed by the game itself, and game source data is aggressively compressed making it unsuitable for further editing. A full snapshot of the source data for a game like Battlefield 4 is likely to be on the order of hundreds of GB, maybe getting up to the TB range. And if multiple teams are working on the same source data, then the source data needs to be sent between them.
Also, before the final few months of development, games' runtime data will often significantly exceed the final shipped size because compression and quality tradeoffs have not been decided and applied.
Much like how software doesn't always ship with source code, games don't ship with source assets.
Normal maps are generated from high-polygon meshes that do not ship with the finished product.
Final textures are the result of psd files with layers, reference materials, alternative versions. These do not ship with the finished product.
Maps are compiled from a source format. E.g. vmf -> bsp. The vmf does not ship with the finished product.
http://www.explosion.com/45404/call-duty-ghosts-pc-install-5...