Hacker News Reader

Comment by itsnotlupus | Hacker News Reader

Is indoctrination even needed at this stage?

Are there openCL equivalents to the popular GPU-accelerated NN frameworks/libraries?

You only need to convince people when they have another choice.

Theano has the infrastructure in place for OpenCL, but not all 'operations' are implemented, which (for any decent calculation) means that it's a no-go [1].

Unfortunately, the focus at the Montreal lab that has a huge influence on its development seems to be (a) 'blocks' for a high-level DNN environment (which is very cool) and (b) CUDA-to-the-max (which is understandable, given Nvidia actively seeds research labs with freeby cards, and -- as evidenced by the article -- is putting a lot of effort into supporting deep learning).

rant start:

It's a shame that OpenCL doesn't get more love. Just the other day there was a cool Clojure GPU project (based on OpenCL) announced on HN. One of the comments was 'will you be building this for CUDA too?'. Rather than pressure open source writers to support closed systems, it would be better to pressure Nvidia to provide up-to-date OpenCL drivers. Newer Nvidia cards are at OpenCL 1.2. And the (somewhat old) OpenCL drivers are always there in an Nvidia install. But does Nvidia ever talk about that : No. It's entirely in Nvidia's interest to encourage everyone to talk CUDA-only. But on a GFLOPs/$ basis, and for the cause of Free, CUDA isn't the right way to go.

rant end.

[1] https://github.com/Theano/Theano/issues/2936

Disclaimer, it's my project, but I run an open source project called deeplearning4j, who's algorithms have a hardware abstraction layer built in to them called nd4j. You get numpy on the jvm and hardware as a jar file. Deeplearning4j itself is built on top of that. Would love to help spread deep learning to different runtimes.

Here's the current work being done on opencl: https://github.com/deeplearning4j/nd4j/tree/master/nd4j-jocl...

We'd love to get this finished. Bit more to do yet though...definitely looking for contributors here. You'll get opencl neural nets for free.

interesting. What's the performance like on the same/equivalent GPU when comparing CUDA to OpenCL?

Need to run empirical benchmarks. CUDA is usually faster. I'd like to run my own benchmarks with nd4j though. We have our own benchmark setup that works for every backend. It allows us to do some interesting things. Cuda itself is usually faster with data transfer latency though[1].

Looking forward to running these ourselves after our opencl support kicks in (only the kernels are written =/)

I plan on basing the work for open cl on our cuda work which is fairly well established at this point (mainly doing optimizations not much change in architecture)

[1]: http://arxiv.org/pdf/1005.2581.pdf

"CUDA is usually faster."

I wouldn't be surprised if this turned out not to be accidental. I mean, it wouldn't work against NVidia for OpenCL to continue to be seen as the "slower option". So I'm sure their efforts to improve their OpenCL implementation aren't considered as important from a business point of view.

True, though if you compare roughly equivalent Nvidia and AMD GPUs, my impression is that the CUDA implementation on Nvidia still outperforms OpenCL on AMD for deep learning. Is this right?

AMD cards are a lot cheaper, would be good to be able to use them for deep learning too - https://www.reddit.com/r/linux/comments/2zgpj8/15000_nvidia_...

And new fast, low power FPGA's from Altera support OpenCL. https://www.altera.com/products/design-software/embedded-sof...

"Is this right?"

Yeah, I think this is the previously mentioned float performance thing.

OpenCL neuralnet backend for the Torch framework. Supports all kinds of stuff.

https://github.com/hughperkins/clnn

I wonder if Vulkan will replace CUDA

Reply on news.ycombinator.com