Nvidia announces CUDA 4.0 | Hacker News Reader

Nvidia announces CUDA 4.0

pressroom.nvidia.com

41 pointsm0nastic26 comments

Been working on the API improvements for this for quite a while now. If there are any questions I can answer, I'll tell you whatever I can.

(edit: in case it's not obvious, I work for NVIDIA on CUDA)

I haven't done any CUDA programming, however I have wanted to dive in. I've been to some conferences where the breakpoint between whether CUDA gives enough benefit to the computation is largely driven by time to load a dataset into memory on the GPU. From what I've been told this has limited the use of CUDA to batch processing on large scale datasets. Is this still a limitation?

PCIe can be a limitation, but there are a lot of ways to amortize the latency of copying data to/from the GPU. Overlapping transfers with kernels, direct load/store from system memory to a kernel, multiple kernels running on the chip at the same time--there are a lot of things you can do. But in general, you're looking at a runtime of at least a few hundred microseconds before you're going to be able to get a benefit from the GPU.

Congrats on your upcoming major release! I'm sure it was a lot of work.

I'm a detail freak so I worry about this kind of thing more than most, but wouldn't a unified address space tend to hide large bus transfers from the programmer?

Will the APIs still let you anticipate where the costs are, or are developers dependent on Windows-only profiling tools?

It's not a true unified address space in the way I think you're getting at. We carve out the address space, but you can't just directly dereference a GPU pointer from CPU code or vice-versa. What we guarantee is that you can determine the location of a pointer based only on the value of the pointer so you can make the appropriate API calls to do a copy if necessary.

If the programmer still has to perform manual copies, what's the advantage?

You can take an arbitrary pointer value in a function, look up its location, and do the copy if necessary, instead of requiring multiple entry points for CPU and GPU pointers. This also simplifies multi-GPU programming a lot--copy from one arbitrary pointer to another and it just works, instead of having to specify which device the copies are going to or anything like that.

One thing that completely puzzles me: How does dynamic memory allocation/deallocation in GPU kernels work? Also, will the programmer be able to use virtual functions in CUDA kernels? If so, does the usual thinking about warp divergence apply? That is--- should all threads in a warp ideally be calling the same function address?

I think there's some preallocated memory for dynamic allocations from the device. As far as I know virtual functions work as well, and the usual thinking about warp divergence does apply.

Do you know if the dynamic allocation pool is per-SM or in global memory?

Global memory.

Have you done any benchmarks regarding inter-GPU memory bandwidth (i.e. GPU1->GPU2) vs. GPU1->host->GPU2? I am working on multi-gpu problems, which are often throttled by bandwidth limitations. Does the new SDK have examples of this and the new unified memory architecture (which looks handy)?

Is anybody who is interested in CUDA also interested in the idea of GPGPU computation in the browser using webGL? Here is a link to a proof of concept I did a while ago http://learningwebgl.com/blog/?p=1828

I think the concept is pretty compelling but I don't get that much traction when I bring it up. Any thoughts?

anandtech has some good info, too: http://www.anandtech.com/show/4198/nvidia-announces-cuda-40

The listed link is PR, but here is also some more details about what 4.0 contains:

http://developer.nvidia.com/object/cuda_4_0_RC_downloads.htm...

Seeing all the buzz about CUDA I can not stop wondering if OpenCL has any future. Does it?

It does, I think. OpenCL's trump card is that it works on non-Nvidia devices as well. I believe that'll ultimately be enough for it to win out, despite the technical merit being in favour of CUDA right now.

If you are interested in GPU coding, checkout www.tidepowerd.com for a .net cross vendor system, also falls back to CPU and they are a startup :)

CUDA has come a long way since it's birth as a student project.

Who wrote the first version? I always assumed it was nvidia's version of Brook.

CUDA and BrookGPU share the same creator, but there's not really anything else in common. The programming models are fundamentally different (multi-level memory hierarchy with sharing of local data with fast synchronization versus pure streaming).

So forgive my ignorance, but would the following be of value in a system design/architecture.

(I dont know what type of cards you have which you would use in this, such as a multi-GPU cored PCI-e card, so in my ignorance, I will assume these exist and are separate from a traditional "video card")

I would be interested to see a system, as a brick, which has a mobo with multiple PCI-e slots, as many as one can get then fill it with both GPU based cards, as well as Fusion IO cards that provide extremely fast IOPs on the storage side. The main CPU in the box, is effectively used to boot the OS - all the other processing and IOPs occur on the GPUs and the Fusion IO cards as data storage and caching...

Is this a realistic idea?

GPUs can't run arbitrary code. You have to code specifically for them. They don't run operating systems or web servers or whatever. These are for massively parallel mathematical computations, not for system calls and threads and stuff like that.

Right, I do know that --

What Iam saying is that a system that uses both GPUs and the fusion IO cards would seem to provide a kickass platform for mass computation on lots of data in a small form factor.

Cuda rack servers are available for multiple vendors.

http://www.geeks3d.com/20091009/cxt8000-nvidia-tesla-based-s...

haha, When I was new to IT in Silicon Valley in 1997, we bought a bunch of clone servers from Colfax International - dual pentium II something-or-others, based on Super Micro mobos.

We were running Windows NT with SQL Server as well as Linux (the founders of Linux Care, Dave Sifry and Art Tide set them up - we were doing EDI with sun, and were one of the first companies to use XML for the docs, which were manufacturing orders for copies of Sun OS -- we physically manufactured all their software (cd, manuals, boxes etc, and dropped shipped them to the customer)

Anyway - after a few months we were having HORRID problems with the colfax boxes always locking up. They were about 12 or 14,000 bucks each at the time - which was insanely expensive for what we were getting. I think they had like a gig or ram or something small.

We also had AS/400s which were many more times that, but that was for the JDE ERP system...

We tried to work through the problems with Colfax, and finally called a meeting with them.

They came in and my manager asks, "Colfax.... Colfax International. Where are your offices?"

CI: "Sunnyvale."

US: "So, any other offices"

CI: "Nope - we are all based in the US"

US: "I see, any international customers?"

CI: "Nope, all in the bay area"

US: "So why do you call yourselves "international""

CI: "...."

--- Anyway, we ended up scrapping the boxes as high end IT workstations and bought name brand systems.

Last time I bought clone HW. We even had some of the first VA Linux and Penguin systems -- when they were barely seen as anything but clones...

Reply on news.ycombinator.com