Hacker News Reader

Comment by inetknght | Hacker News Reader

Look to GPUs for better solutions: make more discrete -- less functional overall but higher performance -- cores, and move them closer to the data; then let the CPU just handle coordination of the discrete processors. Think of SIMD but on a massive scale.

Think of blocks of RAM with math processors. Or the same in your NVMe/NIC/etc.

View on HN

'systems code' like protocol implementations and device drivers tend to be very control-flow centric. on a simd machine in the worst case this means narrowing the 'vector length' to effectively 1. since these are throughput machines, thats often pretty bad.

I do agree with you that lots of little processors is a good way forward here with a careful eye towards reducing sharing of state, but maybe its useful in this case for them to have their own instruction streams.

I thought separate cores do have their own separate instruction streams -- sometimes even completely different architectures and/or supported instructions? Is that not the case?

No, they aren't. Generally you have some level of grouping and complex rules on how and when can they branch

HP announced their ambition to do that a few years ago, framing it as a grand vision of the future of computing. I haven't heard anything since. This kind of offloading or distributed computing isn't quite a new idea, but it hasn't materialized yet. I suspect that it is too tough a nut to crack for the general case.

My understanding is that GPUs have moved toward what I described: thousands of discrete cores with large amounts of math performance but generally terrible (or even non-existent) branching...

GPUs are latency hiding engines...they address the mismatch between processor clock speeds and memory latency by a unique scheme. Since they can’t improve memory latency, instead they parallelize memory and vastly increase the bandwidth available.

Once they have increased memory system bandwidth to be able to feed the multiprocessor throughput, the rest of the architecture is designed to make the most efficient use of it.

They spawn thousands of threads and schedule them in and out really quickly... so the processor utilization is always as close to 100% as possible. When a thread is waiting for memory it is put to sleep in a few clock cycles. When it’s data is available it wakes up and does it’s business. Since the workload is split among thousands of threads therefore there is somebody ready to be scheduled. This is why GPUs only make sense to run on massive workloads.

The same trick can be used to hide SSD latency.

And transferring data to and from the GPU is also a source of overhead.

Despite having an overhead of transferring data to and from the GPU: it's still faster to move the data to the GPU and let the GPU process it locally than it is to leave it in main RAM away from the processing. Transferring data to and from the CPU is also a source of overhead; even your CPU has cache to bring the data more local. So what's your point?

Whatnyounsay is not true as a blanket statement: the overhead for transferring data to GPU memory is higher than accessing main memory from the CPU, so depending on what you do, the data transfer to the GPU might not result in a performance gain at all.

Reply on news.ycombinator.com