My takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild
Data-sizes has continued to grow and HDD-seek times haven't gotten better due to physics (even if streaming probably has kept up), the assumption isn't too bad considering history.
It's a good that they actually revisited it _when they had time_ because launching a game, especially a multiplayer one, will run into a lot of breaking bugs and this (while a big one, pun intended) is still by most classifications a lower priority issue.
I'm not an arrowhead employee, but my guess is at some point in the past, they benchmarked it, got a result, and went with it. And that's about all there is to it.
>We now know that, contrary to most games, the majority of the loading time in HELLDIVERS 2 is due to level-generation rather than asset loading. This level generation happens in parallel with loading assets from the disk and so is the main determining factor of the loading time. We now know that this is true even for users with mechanical HDDs.
they did absolutely zero benchmarking beforehand, just went with industry haresay, and decided to double it just in case.
the "industry hearsay" from two replies above mine is about deliberate data duplication to account for the spinning platters in HDD (which isn't entirely correct, as the team on Helldivers 2 have realized)
This has nothing to do with consoles, and only affects PC builds of the game
So the PS5's SSD architecture was what developers were familiar with when they tried to figure out what changes would be needed to make the game work on PC.
Maybe you're saying the hearsay was Sony exaggerating how bad hard drives are? But they didn't really do that, and the devs would already have experience with hard drives.
If the Helldivers devs were influenced by what Sony said, they must have misinterpreted it and taken away an extremely exaggerated impression of how much on-disk duplication was being used for pre-SSD game development. But Sony did actually say quite a bit of directly relevant stuff on this particular matter when introducing the PS5.
But uh if the devs didn't realize that, I blame them. It's their job to know basics like that.
And on top of any potential confusion between normal SSD and fancy SSD, a mailbox is a super tiny asset and the issue in the spiderman game is very rapidly cycling city blocks in and out of memory. That's so different from helldivers level loading.
Everything else about the PS5 SSD and storage subsystem was mere icing on the cake and/or snake oil.
Do you benchmark every single decision you make on every system on every project you work on? Do you check that redis operation is actually O(1) or do you rely on hearsay. Do you benchmark every single SQL query, every DTO, the overhead of the DI Framework, connection pooler, json serializer, log formatter? Do you ever rely on your own knowledge without verifying the assumptions? Of course you do - you’re human and we have to make some baseline assumptions, and sometimes they’re wrong.
https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence
It was a real issue in the past with hard drives and small media assets. It's still a real issue even with SSDs. HDD/SSD IOPS are still way slower than contiguous reads when you're dealing with a massive amount of files.
At the end of the day it requires testing which requires time at a time you don't have a lot of time.
>wait for the whole list to finish rather than blocking on every tiny file.
And this is the point. I can make a test that shows exactly what's going on here. Make a random file generator that generates 100,000 4k files. Now, write them on hard drive with other data and things going on at the same time. Now in another run of the program have it generate 100,000 4k files and put them in a zip.
Now, read the set of 100k files from disk and at the same time read the 100k files in a zip....
One finishes in less than a second and one takes anywhere from a few seconds to a few minutes depending on your disk speeds.
The Fence is a parable about understanding something that already exists before asking to remove it. If you cannot explain why it exists, you shouldn't ask to remove it.
In this case, it wasn't something that already existed in their game. It was something that they read, then followed (without truly understanding whether it applied to their game), and upon re-testing some time later, realized it wasn't needed and caused detrimental side-effects. So it's not Chesterton's Fence.
You could argue they followed a videogame industry practice to make a new product, which is reasonable. They just didn't question or test their assumptions that they were within the parameters of said industry practice.
I don't think it's a terrible sin, mind you. We all take shortcuts sometimes.
You will be surprised what some people are playing games on. e.g. I know people that still use Windows 7 on a AMD BullDozer rig. Atypical for sure, but not unheard of.
old stuff is common, and doubly so for a lot of the world, which ain't rich and ain't rockin new hardware
Pretending that this is an outrageous decision when the data and the commonly assumed wisdom was that there were still a lot of people using HDDs.
They've since rectified this particular issue and there seems to be more criticism of the company after fixing an issue.
To be fair, the massive install size was probably the least of the problems with the game, it's performance has been atrocious, and when they released for xbox, the update that came with it broke the game entirely for me and was unplayable for a few weeks until they released another update.
In their defense, they seem to have been listening to players and have been slowly but steadily improving things.
Playing Helldivers 2 is a social thing for me where I get together online with some close friends and family a few times a month and we play some helldivers and have a chat, aside from that period where I couldn't play because it was broken, it's been a pretty good experience playing it on Linux; even better since I switched from nvidia to AMD just over a week ago.
I'm glad they reduced the install size and saved me ~130GB, and I only had to download about another 20GB to do it.
They basically just made the numbers up. Wild.
As an aside, I do enjoy the modding community naming over multiple iterations of mods - "better loading" -> "better better loading" -> "best loading" -> "simplified loading" -> "x's simplified loading" -> "y's simplified loading" -> "z's better simplified loading". Where 'better' is often some undisclosed metric based on some untested assumptions.
The wife cuts the end off of the ham before putting it in the oven. The husband, unwise in the ways of cooking, asks her why she does this.
"I don't know", says the wife, "I did it because my mom did it."
So they call the mom. It turns out that her mother did it, so she did too.
The three of them call the grandma and ask "Why did you cut the end off of the ham before cooking it?"
The grandma laughs and says "I cut it off because my pan was too small!"
> The pop-culture cargo cult description, however, takes features of some cargo cults (the occasional runway) and combines this with movie scenes to yield an inaccurate and fictionalized dscription. It may be hard to believe that the description of cargo cults that you see on the internet is mostly wrong, but in the remainder of this article, I will explain this in detail.
FWIW, I meant it strictly in the generic vernacular sense in which I've encountered it: doing something because it has the outward form of something useful or meaningful, without understanding whether or how it works.
Given the problematic history you shared, it seems a new term is needed for this... maybe "Chesterson's Folly"? It's related to Chesterson's Fence (the principle that it's unwise to remove a fence if you don't know why it was erected). If you leave in place all "fences" you don't understand, and never take the time to determine their purpose, fences which serve no good purpose will accumulate.
For their newer instalment, Fatshark went with a large rework of the engine's bundle system, and players on HDDs are complaining about long loading times expectedly. That game is still large at ~80GB, but not from duplication.
[1]: https://www.reddit.com/r/Vermintide/comments/hxkh0x/comment/...
It was a fundamentally sound default that they revisited. Then they blogged about the relatively surprising difference it happen to make in their particular game. As it turns out the loading is CPU bound anyway, so while the setting is doing it's job, in the context of the final game, it happens to not be the bottle neck.
There's also the movement away from HDD and disc drives in the player base to make that the case as well.
> multi minute load times
23Gb / 100mb / 60s = 3.92m
So in the worst case when everything is loaded at once (how on a system with < 32Gb RAM?) it takes 4 minutes.
Considering GTA whatever version could sit for 15 minutes at the loading screen because nobody bothered to check why - the industry could really say not to bother.
Instead they did blindly did extra work and 6x’ed the storage requirement.
I don’t know about the Xbox, but on PS4 the hard drive was definitely not fast at all
>we looked at industry standard values and decided to double them just in case.
it had no serious or glaring impact to their bottom line.
thus it was the right call, and if they didn't bother to fix it they'd still be rolling in $$$$
It will make them a lot of money and is thus the right call. Who cares about customers am I right? They'd still be rolling in $$$$.
Games would be much better if all people making them were forced to spend a few days each month playing the game on middle-of-the-road hardware. That will quickly teach them the value of fixing stuff like this and optimising the game in general.
That’s how we wound up with this game where your friends are as much of a liability as your enemies.
Pay 2000$ for indie games so studios could grow up without being beholden to shareholders and we could perhaps get that "perfect" QA,etc.
It's a fucking market economy and people aren't making pong level games that can be simply tuned, you really get what you pay for.
In my last project, the gameplay team played every single day.
> Games would be much better if all people making them were forced to spend a few days each month playing the game on middle-of-the-road hardware
How would playing on middle of the road hardware have caught this? The fix to this was to benchmark the load time on the absolute bottom end of hardware, with and without the duplicated logic. Which you'd only do once you have a suspicion that it's going to be faster if you change it...