25 Gbit/s HTTP and HTTPS download speeds | Hacker News Reader

25 Gbit/s HTTP and HTTPS download speeds

michael.stapelberg.ch

22 pointssecure12 comments

I wonder what is causing the bottleneck in the TLS case, particularly with kTLS where the CPU is only at 2% but the throughput remained the same. Perhaps there is a limited size stream buffer somewhere in the crypto side that doesn't allow high throughputs whereas the raw networking buffers are more dynamic? I know the Caddy guys read HN so maybe they can chime in with some actual knowledge :).

It might have maxed out the CPU core on the peer/client? There's no mention of what it's CPU usage is.

No, it didn’t max out CPU (not even a single core) on either of the machines.

My theory is that you only have a limited time window per TCP packet, and decrypting TLS exceeds that time window. I haven’t verified that theory yet, though.

But it doesn’t say what TLS cipher suite it is using, making it a pointless benchmarks

I'd be surprised if it matters. In older TLS versions (the version is not specified either) the suite might include various handshake parameters, but they're irrelevant after the handshake completes anyway. After that, regardless of version, realistically you are doing AES or maybe ChaCha20. There are lots of options in TLS 1.2 and earlier, but you won't use any of them, since both machines know AES.

The fact it's bottlenecked somewhere but CPU is fine suggests some configuration parameter somewhere results in not as much data being in flight for HTTPS, limiting throughput over this very fast connection. If so, fixing this parameter would make the noticeable difference essentially vanish.

Maybe, maybe not, since it is a custom configuration can be anything. A coffee lake CPU should be capable of 25Gbps of AES-GCM for a single connection. Not so much for ChaCha20-Poly1305. There is a significant difference.

ChaCha20 vs AES matters a lot, because the latter is hardware accelerated on all important platforms. I did some benchmarking on this in the last year, and e.g. QUIC transfers on the same machine peaked at 520MB/s throughput per core using AES128, vs 350MB/s using ChaCha20 (see also https://github.com/rustls/rustls/issues/509).

I’m using the defaults of each server.

In practice, as you suspected, this amounts to AES, as per the curl -v output:

SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

Thanks, you should use AES-128-GCM, it is about 40% faster and is much more common on the internet.

The leading "25" got stripped from the title, unfortunately.

Thanks, fixed now. I could have sworn it showed up before I clicked submit.

Sorry, HN's title rewriter made a mistake there. Thanks for fixing.

Reply on news.ycombinator.com