Hacker News Reader

Hacker News

Comment by threeducks | Hacker News Reader

Parent Full thread

threeducks

A few things come to mind:

- Games (400GB for Ark, 235GB for Call of Duty, 190GB for God of War)

- LLMs (e.g. DeepSeek-V3.2-Exp at 690GB or Kimi-K2 at 1030GB unquantized)

- Blockchains (Bitcoin blockchain approaching 700GB)

- Deep learning datasets (1.1PB for Anna's Archive, 240TB for LAION-5B at low resolution)

- Backups

- Online video processing/storage

- Piracy (Torrenting)

Of course you can download those things on a slower connection, but I imagine that it would be a lot nicer if it went faster.

View on HN

> 400GB for Ark

Ark is a strange case. It compresses very very well. Most of it ends up with compression ratios of around 80%.

> Total size on disk is 628.32 GiB and total download size is 171.42 GiB.

From SteamDB's summary of Ark's content depots.

I have 1Gbit at home, but almost never reach those speeds when downloading games. It’s one of those cases where it makes sense (I want to play now!), but I’m under the impression the limit is upstream (at steam most likely), rather than on my connection. (I do get those speeds on speed tests, doesn’t seem to be my setup).

ISPs happily collaborate with and put speed test servers in privileged locations on their network so you will get higher speeds there even if the actual peering to the outside world is much slower.

As I was typing this it came to mind. Will test against one of my own servers one of these days to confirm.

You can try Fast.com (Netflix) or Cloudflare’s one which are explicitly designed to work around this by serving the test data from the same endpoints the serve actual customer data, so ISPs can’t cheat.

This still doesn’t guarantee however that you will achieve this speed to any random host on the internet - their pipe to Cloudflare/Netflix may very well be fat and optimized but it doesn’t guarantee their pipe to a random small hosting provider doesn’t go over a 56k modem somewhere (I jest.. but only a bit).

Given that whether you get 30mbit or 30gbit from Netflix won’t make a blind bit of difference it’s not that useful a test. It doesn’t do upload either as Netflix is all about consumption.

Test to where you want to exchange high speed traffic.

Fast.com does an upload speed test, but it's hidden behind the "Show more info" button.

I get full speed on steam downloads, even set the limit lower so youtube doesn't buffer.

Steam is tricky cause it has multiple potential bottlenecks. The steam cache, internet connection, decompression (i.e. cpu) and storage. Often hard to tell which limit you're hitting

I have 5 gigabit and usually get ~1.2 gbps, sometimes get up to ~2 gbps from Steam.

You might check what region Steam is downloading from (it's in settings -> Download or something similar). If it's selected poorly, you might do better by picking one yourself.

I get about 2Gbps max on Steam and Xbox on an 8Gbit connection. The limit could also be due to your disk drive while Steam is installing the downloaded files.

Is the connection the bottleneck for 700GB?

Reply on news.ycombinator.com