Hacker News Reader

Comment by exDM69 | Hacker News Reader

CUDA (and OpenCL) C is just like C, with the exception that recursion is not allowed because there is no runtime stack to support it. Instead, there's just a lot of registers.

However, the main difference is the number of different types memories there are available. That makes GPU programming very tricky indeed. There's around 6 different types of memory visible to the programmer, each with it's own distinct access times and restrictions. On the host side (CPU), there's regular memory and page-aligned DMA buffers. On the device (GPU), there's constant, global, local and texture memory. Most of the time it's clear which memory you should use, but occasionally it takes some thinking. Especially deciding whether something should go to texture memory or global memory can be difficult without doing it first and benchmarking.

View on HN

whoa: recursion has been supported since Fermi support was released in CUDA 3.0 (there's a stack pointer and a stack frame and everything). what's not supported until GK110 (Kepler 2) is GPU kernels launching/waiting on GPU kernels.

generally (and especially in the case of the upcoming GK110 chip), you should use global memory. GK110 improves this with LDG, which allows you to get some caching benefits of texture (spatial locality) without having to jump through the API hoops required to use textures.

(full disclosure: I run the CUDA driver team at NVIDIA)

> whoa: recursion has been supported since Fermi support was released in CUDA 3.0 (there's a stack pointer and a stack frame and everything). what's not supported until GK110 (Kepler 2) is GPU kernels launching/waiting on GPU kernels.

Oh, cool! This opens doors for applying GPGPU to a whole new class of algorithms. I clearly must update my GPU knowledge.

Global mem vs. texture mem is always the biggest choice. Sometimes it's worth thinking whether there's a potential win in caching texture/global memory fetch results in local memory. So there's still choices to be made, even though the hardware has become better and easier to program.

With Fermi and newer (i.e., Kepler) hardware, there's not as much to be gained by using texture memory. On previous hardware, the texture cache helped to speed up some kernels that had non-uniform memory access patterns; the Fermi/Kepler hardware has a larger on-chip L1/L2 cache which serves the same purpose and does so without requiring the programmer to write extra code for working with textures.

CUDA 5.0 (not released yet) is going to have recursion and a bunch of other things. But most of these new features are available only on the new kepler 2 devices.

As for memories, it becomes natural once you start programming CUDA day in and day out and understand the algorithm well enough :)

well there are more types :) shared memory: which is reserved for the SM (a bunch of CUDA cores) so that cores in the same block (SM) can share data. pinned memory: you can disable OS paging in the RAM (since CUDA operates in the driver level)

> well there are more types :) shared memory: which is reserved for the SM (a bunch of CUDA cores) so that cores in the same block (SM) can share data. pinned memory: you can disable OS paging in the RAM (since CUDA operates in the driver level)

I used different (OpenCL-like) terminology, but we're talking about the same things. Shared memory <=> local memory. Pinned memory <=> page locked dma buffers.

i thought "local memory" refers to thread-local memory, i.e: registers (you didn't count this elsewhere)

and i am not sure "page aligned" and "page-pinned" are the same thing.

Reply on news.ycombinator.com