After decades of being the principled underdog, hopefully everyone can rally around them and make sure their open source projects work well with AMD products and contribute to their new open source initiatives.
But these days AMD is a serious contender at all levels! If the performance or value isn't actually in AMD's favor, it's near enough to be undetectable outside of a benchmark, and very easy to support the good guys.
But AMD doesn’t run CUDA? Is it very easy to switch a stack relying on CUDA to OpenCL? I don’t think so.
How well it works is likely codebase and application dependent.
The added code complexity was not worth it - we originally did it as NRE for a big client who wanted OEM capabilities to our IP.
https://arxiv.org/ftp/arxiv/papers/1005/1005.2581.pdf
In basically every chart and graph, OpenCL is slower.
OpenCL has a higher level of abstraction, so it's got a higher penalty for it. But, you get cross-platform support. nVidia doesn't optimize for OpenCL either because they don't want you to use it over their own competing framework.
But 'dats cool, keep downvoting, guys. Sources aren't real if you can downvote 'em enough. God, I love Hacker News.
Most have some kind of branch or patchset with OpenCL support. The problem is that they aren't great. If you need any new layers there is no support. There is nothing like CuDNN so you don't get the high speed convolutional kernels.
It's great to blame developers for supporting NVidia, but the thing is NVidia are great to work with. They dedicate large teams to deep learning support (not like the 2 or 3 part time Devs AMD does), and they publish good research and tutorials. AMD does nothing like this.
AMD has done some interesting work on HCC (a more proper gpgpu compiler approach with llvm base) and that is showing promise. See here: https://instinct.radeon.com/wp-content/uploads/sites/4/2017/...
Additionally, they support cuda decompiling to HIP which is an intermediary that can be built to target Nvidia (via nvcc) or amd (via HCC).
Nvidia has built a lot of tooling for DL, such as cudnn, and the new Tesla cards have dedicated silicon for tensor calculation. Amd does have a cudnn equivalent called MlOpen. They have also ported caffe via HIP and it works well. Work is being done by amd right now to torch, mxnet and tensorflow to add support for amd hardware with minimal burden to the maintainers of these projects.
You can read about some of the DL toolkits available here: https://instinct.radeon.com/en/6-deep-learning-projects-amd-...
I think it's particularly bad form on behalf of everyone in the DL framework and library world to cator only to Nvidia and cuda, and that they very much walked into this shake down with open arms.
The original comment is correct in that contributing support for OpenCL (which works on mobile too) will alleviate this to a fair degree. It's one of those things where the more momentum is behind it, the more device manufactures will focus on ensuring their opencl compiler is building properly optimized kernels for their hardware.
Start contributing to OpenCL or adding hip support to existing projects and we'll see some viable alternatives pop up from not only AMD, but players like Qualcomm and Samsung.
I'm not saying it's a great situation, I'm saying that NVidia has always had better libraries, tools and performance, and it isn't surprising that developers use them.
Deep learning is hard and slow enough without using second class tools.
That's an order of magnitude worse performance than NVidia on ResNet 52 for 2 months with no real reason.
Great idea, but no one can use it reliably yet.
http://vertex.ai/blog/tile-a-new-language-for-machine-learni...
Two of the big motivators for opening the code were 1) giving students taking the popular courses a way to get started with GPU in whatever machine they've got (recent Intel GPUs in say a MacBook Air are enough) and 2) giving researchers a platform where it's simple to add efficient GPU-accelerated ops.
For scale on #2 check out the entire implementation of convolution:
https://github.com/plaidml/plaidml/blob/master/plaidml/keras...
That said we'd be pretty excited if someone wanted to add support for TF, PyTorch, MXNet, etc. We like Keras but are happy to have integrations for all frameworks. With work you could pair it with Docker and containerize GPU-accelerated workloads without the guests even needing to know what hardware it's running on. Lots of possibilities.
> whatever penalty Keras might impose is equal in the two cases.
The penalty Keras imposes when using Tensorflow depends on its Tensorflow implementation. The penalty Keras imposes when using MXNet depends on its MXNet implementation. The penalty Keras imposes when using PlaidML depends on whatever the PlaidML devs implemented. When you build a Keras layer, it's calling different Keras code for each backend.
The comparison would be fair if Plaid claimed to be the fastest Keras backed, not if it were actually claiming to be faster than Tensorflow.
I think they have a lot of challenges ahead of them, but I’m still more optimistic about Plaid than AMD’s own efforts.
AMD says that they don’t care about ML[1], and their actions back that up.
Edit: and to be clear, I think comparing Keras+Plaid vs Keras+TF is an entirely valid thing to do. Lots of people work in Keras, and if you download a random NN code off github it likely to be Keras (or Pytorch now of course).
[1] https://www.reddit.com/r/MachineLearning/comments/66bgmf/com...
It's a patchset (note that it isn't upstreamed in their chart), and it's caffe. That was great in 2015.
But then it got popular outside of research labs too, so Nvidia has decided to milk the cow and restrict this feature only to their Quadro line-up, with ATI following suit. The result has been that they have pretty much killed the market - apart from those research labs nobody is going to buy a $3-4k Quadro card that has otherwise the same performance as a $300 GeForce only for the stereo support.
I think this sort of artificial crippling/restricting of usage to force industrial customers to use the more expensive hw they don't need otherwise will only lead to a proliferation of task-specific ASICs and Nvidia will hurt only itself with it. The reason why people use Nvidia GPUs for parallel computing is cost and ubiquitous availability, not because there aren't other options. This move will only accelerate their development - for which there were no reasons until now.
There's also just not much incentive for people to move to AMD here. There's a reason NVIDIA gives out these graphics cards like candy to academia. It's because they want to make their margins in data center while keeping the broader developer community locked in to cuda.
People on HN make broad and sweeping comments about "if you open it they will come and everything will be magically better".
It's a lot more complicated than that. The market incentives just aren't quite there yet. Could it happen one day? Yes. Will it happen today? No. It's going to take a lot more than this for other vendors to start supporting AMD.
What I will say: There will be competition and the space is heating up. AMD is one player.
Now let me put my commercial hat on here: What would it take for me as a deep learning vendor to support/care about AMD?
1. Show me the money. I need a clear revenue stream. AMD has some hope here. Customers don't care about which gpu they use as long as it fulfills a use case they care about.
2. Code: Show me something that's actually a robust. A 1 off fork of any deep learning framework (see the random 1 off caffe forks that aren't actually caffe by nvidia, intel,amd)
3. Share the burden. Put the code out there and support the broader community. Actually maintain and follow up with the latest innovations (hint: this is hard. throwing code over the wall once doesn't mean anything)
4. Amazon will be key here. Google cloud (look google is great but they aren't the leading cloud player by a long shot and likely won't be anytime soon) - get them on board with some AMD cards.
5. (Disclaimer I know the founders) - make opencl not suck. https://vertex.ai/ is an interesting player in this space.
So look, I won't say it's impossible. Let's just not ignore the actual commercial market forces that are also at play here.
What is Cisco is the current question.
There's always big companies that get disrupted because they kill good will. That's the start to something. It takes more than 1 move. Will NVIDIA make this mistake over time? Likely.
Will it be AMD and Tensorflow support that does it? Not by a long shot. It will be by a series of players providing the right incentives. My response (nor any response to this) should not be treated as binary. Ultimately in order for a company to be "disrupted" you need to actually analyze and exploit the market forces involved here. just writing this off as "open ecosystem is all we need" is dangerous.
The other wildcard i haven't seen people mention here is the Neural Network Processor (NNP) from Intel Nervana. The hardware has potential. As long as the software doesn't force us to use BigDL, it has potential.
Re: AMD/Intel. We've been waiting for them to get their act together for years. Nervana could be great but I'm going to wait on that one. So far their "launches" have been nothing more than marketing fluff.
As for your projections about tensorflow. It won't be tensorflow. Tensorflow will be 1 of many frameworks. Look I like HOPS but you guys push tensorflow explicitly. A startup running its own hadoop distro that happens to push tensorflow isn't going to move the needle. You guys are great middleware I'm sure but I haven't seen the customers where it might be viable. I hope you guys continue to grow though! While you're doing that, most hadoop vendors are focused on moving up the stack. It will take people with actual resources to move the needle in terms of enterprise adoption.
Amazon is doing this with mxnet and EMR, MapR is pushing tensorflow in their serving. CNTK is being pushed in SQL server and HDInsights. There's some competition there.
What I'm getting at here is: it will take multiple vendors and competition. I'm going to place my bet on the bigger players already involved with the foundations first though. Open standards (addressed below) where it commoditizes the chip will be key. The storage infra will follow from that. It should be something that doesn't displace current infra but allows interop.
Things like nvvm from the mxnet folks, onnx (where the framework doesn't matter anymore!) being pushed by the various hardware vendors etc will move the needle. You need buy in from the actual big players who can upfront the development time in to making these things viable alternatives.
For your "seamless transition" I'm not sure that would be that hard done right. Supporting "great tensorflow" can come in multiple flavors. As a separate issue, tensorflow's production story is horrible. That's another topic I could rant about all day though. It ultimately comes from abstracting it away though. That by itself is a hard problem (Disclosure: I have my own solution for this that I won't talk about here just know I'm biased :D)
Lastly, I question whether opencl can even be a viable alternative. It's a fragmented inconsistent standard with a worse API than cuda. One reason it "won" is because it's in general cleaner and a clear leader in the space.
A $190 billion company earning 4x what nVidia does, a mere $9-$10 billion in net income, with 12x the cash of nVidia. A gigantic money printing machine, is what it sounds like they are.
What's wrong here with many of the guesses people are making though: They think simple open standards are enough.
It's a lot more than that. Don't alienate the market forces that also need to move to make this work. Too many coders write off the real business incentives side of this.
Viable competition will come from multiple vendors and possibly backwards compatible/interchangeable standards that don't require many changes in code. Give me a path forward as a vendor. Show me the money. Put code out there and maintain it/keep it up. Show me it's going to stick around.
It's like any SAAS you'd pay for or an IDE you invest in, vendors want to see an ecosystem that fulfills a set of requirements and allows them to get work done for their customers.
Right now cuda is still that best tool. I covered what will likely need to be considered in my parent post so I won't reiterate that here.
I only ask that folks don't assume that "open source linux drivers and open standards and 1 open source framework with opencl support" are enough to unseat nvidia. The market won't move for that.
have a look, people had to make a petition to ask AMD to fix their craps.
https://www.phoronix.com/forums/forum/linux-graphics-x-org-d...
Obviously the petitioner is also clueless, or else how could they write "We feel that Radeon hardware is vastly superior to competition", when their hardware in 2013 was nowhere near NVIDIA. (See the review the petition links to for an example...).
You sir are a typical phoronix reader. Uninformed, biased and thick.
If you're involved in the tvm project - are you aware of any major deep learning libraries working on utilizing it? I'd imagine MXNet would be the first one to try it, but I haven't seen anything.
And while slightly off topic, their CPUs contain just as much Evil as Intel's CPUs (PSP/ME). I wouldn't be so quick to call them a "principled" underdog.
1. restrict its software to be deployed at one physical or virtual computer.
2. Similar to one but restrict it to be deployed the specific vendor hardware.
3. Non-commercial usage only
4. Commercial usage allowed but only for small company less than 50 employees, buy more expensive license for that.
What's the difference?
If those ridiculously existing consumer-unfriendly licenses are valid, I think NVIDIA's stupid no-data-center-deployment-except-for-blockchain-processing license is also valid.
But AMD/ATI cards are heavily bought by crypto miners, so it's almost impossible to buy one, even at an insanely high prize. And NVidia cards are compatible with all games made between 2000 and 2017. AMD driver were very bad several years ago. So if you are playing GoG games or older games occasionally there is only NVidia. Yes, both NVidia with their spyware-driver and now this shit, and AMD driver sucks. For CPUs everyone suggests now AMD which are now better than the crippled Intel CPUs with their plastic pads inside the CPU so that a new Intel CPU will last only about 3 years.
I read that AMD cards were friendly to Linux, so after removing my NVIDIA card for stability reasons, I switched to AMD only to have other issues.
I’m still looking for an easy Debian compatible graphics card so I can do CAD (Onshape - OpenGL). The AMD card works now, but I don’t know what to do next time around.