Comment by dragontamer | Hacker News Reader

Comment by dragontamer | Hacker News Reader

Parent Full thread

> If I need to crunch large amounts of data in a hurry, I'll send it to a GPU. The CPU has no chance to compete with that.

It takes literally 1 to 10 microseconds (10,000 nanoseconds) to talk to a GPU over PCIe.

In the 40,000 clock cycles, you could have processed 2.5 MB of data with AVX512 instructions *BEFORE* the GPU even is aware that you're talking to it. Then you gotta start passing the data to the GPU, the GPU then has to process the data, and then it has to send it all back.

All in all, SIMD instructions on CPU-side are worthwhile for anything less than 8MB for sure, maybe less than 16MB or 32MB, depending on various details.

----------

That's one core. If your one-core machine can talk to the other 32-cores or 128-cores of your computer (see 64-core EPYC dual-socket computers), you can communicate to another core in just 50-nanoseconds or so (~200 clock penalty), and those other cores can be processing AVX512 as well.

So if you're able to use parallel programming on a CPU, its probably closer to 1GB+ of data before it actually is truly an 'obvious' choice to talk to the GPU, rather than just keeping it on CPU-only side.

---------

Example of practical use: AES-GCM can be processed using AVX512 in parallel (each AES-GCM block is a parallel instance, the entire AES-GCM stream is in parallel), but no one will actually use GPUs to process this... because AES-instructions are single-clock tick (or faster!!) on modern CPUs like Intel / AMD Zen.

That's just going to happen whenever you go to an TLS1.2 or HTTPS instance, which is pretty much all the time? Like, every single byte coming out of Youtube is over HTTPS these days and needs to be decrypted before further processing.

Is it safe to say that the true future is low latency GPUs? :)

The future is blurring the line between what a GPU and a CPU is.

> It takes literally 1 to 10 microseconds (10,000 nanoseconds) to talk to a GPU over PCIe.

The CPU has an overhead of about ~10us to enable the AVX512 units.

It also dramatically reduces the clock on other cores.

For more information, see: https://stackoverflow.com/a/56861355

Timing information: https://www.agner.org/optimize/microarchitecture.pdf

Skylake-X is a processor that's 7 years old. Intel's first implementation always kinda sucks, but the newer implementations have no such restrictions.

Its all about AMD Zen4 or Xeon Ice Lake+, which has no clock reduction and no overheads.

From microarchitecture.pdf

On Alder Lake (pg 172).

> The reader is referred to the timings for Tiger Lake and Gracemont.

On Tiger Lake (pg 167):

> Warm-up period for ZMM vector instructions

> The processor puts the upper parts of the 512 bit vector execution units into a low power mode when they are not used.

> Instructions with 512-bit vectors have a throughput that is approximately 4.5 times slower than normal during an initial warm-up period of approximately 50,000 clock cycles.

I'm not saying you are wrong. I just haven't heard about that.

https://www.mersenneforum.org/showthread.php?p=614191

> Since 512-bit instructions are reusing the same 256-bit hardware, 512-bit does not come with additional thermal issues. There is no artificial throttling like on Intel chips.

At least for Zen4, there's no worries about throttling or anything really. Its the same AVX hardware, "double pumped" (two 256-bit micro-instructions output per single 512-bit instruction). But you still save significantly on the decoder (ie: the "other" hyperthread can use the decoder in the core to keep executing its scalar code at full speed, since your hyperthread is barely executing any instructions)

This is why on-die crypto accelerators make so much sense.

Hopefully they're updated for the new post-quantum algorithms.

Which would you rather have: some fixed-function unit shared between all cores (load balancing? what if you're suddenly doing crypto stuff on many cores?), or the general-purpose tools for running any algorithm on any core?

AES isn't really threatened by quantum computing AFIK

And most encryption is to use "something" to get an AES key and then us that to decrypt data.

And that "something" (e.g. RSA/ECC based approaches) is what is threatened by quantum computing. But it's also not overly problematic if that "something" becomes slower to compute as it's done "only" once per-"a bunch of data".

AFIK the situation is similar for signing, as you don't sign the data itself normally but instead sign a hash of it and I think the hashing algorithms mostly used are not threatened by quantum computing either.

> AES isn't really threatened by quantum computing

NSA: "You are correct!"

hm, which AES? AES-128 is getting a bit tight already for multi-target attacks.

As to quantum, it looks like practical serial or parallel application of Grover's algorithm might still be decades away. But that is with current knowledge, and who knows what other breakthroughs will be made.

Wrt algorithms, it really is an implementation detail of how flexible they make the crypto engines.

The number of crypto units would be SKU specific depending on the workload. A server box doing service mesh would need one per concurrent flow presumably.

The thing that accelerator offload gets you is an additional thermal budget to spend on general purpose workloads. If you know that you will be doing SERDES, and enc/dec, offloading those to an accelerator frees up watts of TDP (thermal design power) to spend other places. This is also why we see big/little architectures, the OOO processors suck up a lot of power. In-order cores are just fine for latency insensitive workloads.

Most crypto beyond the initial key exchange is symmetric (AES, ChaCha) and those are still resistant to quantum attacks (well beyond the brute force search speed ups). Post-quantum key exchange is fast enough that it doesn't need dedicated units, unless you're in some super constrained environments. But in that case you'll run into other issues too.

Reply on news.ycombinator.com