Hacker News Reader

Any chance your workload would see a benefit from porting the code to a GPU?

I work on tech to automate that. The generic answer is yes as long as you overcome the PCIe bottleneck (or can hide the latency).

I assume that should be "and can hide latency"?

While GPU offloading is extremely powerful in terms of compute, you have to deal with both bandwidth limitations (de facto ~12 GB/s for x16 PCIe 3.0) and the latency of launching compute kernels and waiting for them to complete.

That depends on the problem space. For some problems latency does not need to be an issue, if the problem can be pipe-lined then you can keep all your cores running at a high percentage of their theoretical maximum load even if latency is high.

For real time applications this is usually not an option, there latency > some threshold will kill your proposed solution.

In those cases where the latency is not a problem, it tends to be precisely because you can hide it e.g. via pipelining :)

You overcome the bottleneck by doing as much as possible on the GPU. Even things you don’t think are fully suited to it.

Reply on news.ycombinator.com