Hacker News Reader

Comment by taeric | Hacker News Reader

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are.

Heck, just the geometry is probably far larger than folks anticipate.

View on HN

> This feels wrong. In that, I'd be surprised if they are not using compression.

Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ...

from https://github.com/IridiumIO/CompactGUI

Thinking more on it, I am not too shocked that they would use uncompressed assets "at rest" on the machine. This is especially true for games mailed out on disks or cartridges. I'm still a little surprised, so the links are neat to read on.

I am kind of curious on why they wouldn't use compression in transport, as it were. I'm still mostly assuming that they do.

Yeah I don't understand why they don't just decompress the audio into RAM as the map loads. Or even just to some swap space.

This list appears to be largely indie games, many with sizes under a gigabyte. I'm not sure why at-rest compression would be a high priority for a small team when their game fits on a $5 thumbdrive and will be compressed during the install process by Steam or Itch.

You can sort by the number of submissions to see more popular games.

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc...

People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.

Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.

Then you have to have the game reprocess all the assets on the users machine which could easily take an hour

Essentially any modern image compression format is going to be limited, or close to, by storage speed (for decompression). So unless the user has insanely fast internet (and the servers are also able to saturate their connection), it will be faster in the end.

With the size of games nowadays, that's likely a net time save for many people from the discounted download time.

Gamers being gamers though they’re not going to say “well, we on reflection I’ve not had to wait so long”, they’re going to slam you with one star reviews for the fact there’s an hour long loading screen on first start.

The recompression/decompression logic could (and I would argue, should) be packaged separately for distribution platform instead of being embedded as part of the game's logic. In the case of using Steam, it would be no different than how it shows separation of time spent downloading vs time spent installing.

This allows it to be ran automatically after download without having the user need to launch the game.

You can combine the approaches! KTX2, for example, supports "supercompressing" GPU block compressed formats with LZ4 or Zstandard. The project I have been working on ships BCn textures compressed with zstd and it has been pretty dang good.

You can add lossless compression, like Kraken, on top of that. PS5 has hardware accelerated Kraken decompression. And you can encode textures for GPU in a way that is better compressable with lossless algorithms.

Hardware accelerated is still not zero cost. Plus if you are publishing for the other console and/or PC there is a question as to whether you want to do the extra work for one platform.

This is where using something like unreal shines. You get the correct default for all the platforms.

That is a little surprising to me. Especially if we are talking about games that are delivered online. Thinking on it, I'm actually only surprised for online delivered games. I don't see any incentive to reduce size on delivered disks.

I'd expect that they use a compression that is good for transport, at transport time. I'm not too shocked that they have the assets in a way that is easy to use at runtime.

Games are duct taped together at last minute on impossible deadlines pretty much every time.

This is exactly it - and once released there is rarely the motivation to go back and optimize side things. The only game I know that has done anything like that is Factorio, or the open source reimplementations of various engines.

The crackers who remove copy protection often repackage (or someone else does) to greatly reduce size; since they’re in there poking around the code they can see what isn’t used.

This isn’t new, either, ancient console games have unused assets and even entire levels in the code but disabled.

Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.

Even for transferring the game to local?

I can see some benefits to having uncompressed assets in that it would help with asset loading. I'd still be surprised to not have any form of compression at all happening. I'd also be curious to know just how much the assets could be compressed.

> It's not just speculation.

So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades.

Why are new games so large? Good question. This entire thread is speculative.

Um, no. It's not speculation. Many games have been shown to use uncompressed assets, that's an objective fact.

The thread might be speculating about _why_ they do that, or whether this particular game does that, but that's a non sequitur, that isn't what I was talking about.

Your statement that some games in the past have used uncompressed assets is a non-sequitur.

Has any game in the past 5 years that has an install size >100gb used uncompressed assets? I would be very very surprised! The likelihood of this being the case on PS5 or XSX hardware is approximately zero.

Yes I’m familiar with the infamous uncompressed audio in Titanfall.

Hitman would be very notable example. Fortnite almost got to 100 gigs when they enabled some kind of asset compression.

Hitman was a special case of bundling older games designed designed for spinning disks.

Fortnite didn’t publish what they did. But I promise it’s not “bEnableCompression=true”. Compression is used up and down the stack with the utmost of care. Even the older games that didn’t use compression did so only after great consideration of the trade offs.

“Games as a Service” games like Fortnite, Destiny, and Call of Duty have often bloated and then been optimized. The fix is never bEnableCompression. It’s more about how the content is sliced and packaged. How to efficiently serve new content is a really tricky problem. And a somewhat new problem that didn’t exist 10 years ago. At least for consoles.

Jedi Survivor having uncompressed textures is a speculation, but I'm extremely confident that it is this way. The fact that games nowadays ship without LOD systems, without texture compression, without adequate streaming is not just speculation. It is a fact.

You're saying Jedi Survivor doesn't use block texture compression? On which target? All targets?

Can you capture a trace in RenderDoc to prove this? Or show raw bmp/uncompressed dds files on disk?

By texture compression I meant something like LZ4 or Kraken on top of block texture compression, sorry for ambiguity. Of course they use block level compression. But it seems that it's the only thing they use, as I said in my top level comment.

You'll find that's pretty common since block compression already reduces sizes pretty effectively. At install time the files will be compressed further over the wire before being decompressed, so I imagine the payoff is not that big in many cases. I do wonder how much of it is that a Kraken license costs money and it takes time to integrate, though...

>already reduces sizes pretty effectively

8k texture with mipmaps takes about 100mb with BC7. Which is huge. Should be around 30-40mb with kraken and oodle texture.

> I do wonder how much of it is that a Kraken license costs money and it takes time to integrate, though...

Kraken license doesn't cost anything for Unreal developers. And Sony licensed it for all PS5 games.

The geometry would be hard to compress, but I suspect there's some clever tricks with some kind of fractal compression[1] that could drastically reduce texture sizes for camo, foliage, etc. without any effect on aesthetics.

Encoding is the hard part, but only need be done once when building the final release.

[1] https://en.wikipedia.org/wiki/Fractal_compression

Wait, are there more papers on this? Seems interesting.

When I was messing around with this (~1995?), there was a Yuval Fisher paper floating around which did a good job of explaining it. Although it looks like it now costs ~$40 to get that paper or ~$100 for his book. Helpful!

> there's some clever tricks with some kind of fractal compression

It's a shame that IFS hamstrung the adoption of fractal compression with their expensive license fees - wouldn't necessarily have been The Future Of Images but at least we'd have had the option.

Reply on news.ycombinator.com