Hacker News Reader

Comment by Talyen42 | Hacker News Reader

People are making fun of facebook/google for expanding beyond core competency, but this is a market (AI training / inference) where Nvidia has a very real monopoly that facebook and google must rely on for core competency products, so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer and partnering with TSMC (or whoever) to make training and inference chips. It's absolutely the right decision today, and if there was one viable alternative to Nvidia (which AMD and Intel are not), it wouldn't be necessary.

View on HN

The entire reason that AMD hardware isn't considered a viable alternative to NVIDIA is that it doesn't run CUDA, which is going to be true of these custom chips too.

The chips FB are working on are probably far more similar to Google's TPUs than anything NVIDIA makes though.

For researchers, yes. At the scale Facebook and Google operate at the lack of CUDA isn't much of a problem. The big difference is that AMD's GPUs only work on vectors whereas Google's TPUs (and NVidia's latest card) can work on matrices directly for a large increase in throughput.

Most DL researchers don't care about CUDA, they use higher level frameworks (Tensorflow, Pytorch, Caffe, etc). The frameworks can be extended to support other backends (e.g. OpenCL). If someone builds their own hardware, provides integrated support for that hardware in Tensorflow, and demonstrates speed advantages, people will use it.

This is something that I've wondered about. There is nothing about Tensorflow (for example) that is specific to CUDA. Why doesn't AMD dedicate a couple of engineers to port it? They don't even have to do a complete job. Only about half of the Tensorflow operations are used for most models.

AMD does. https://github.com/ROCmSoftwarePlatform/tensorflow

If they make CUDA the industry standard by porting it to AMD, and then Nvidia is still able to control the standard, over the long term AMD will always be in the position of having to catch up.

I think the GP is suggesting AMD port TensorFlow to OpenCL to break up the CUDA monopoly.

No need, there is already an industry standard that works across all hardware.

Why do you say that? Because they already have a large code base that would be too much work to move over? Or superior features in CUDA? This is actually related to some ongoing research for me, be very interested in any insight.

Because there's no compelling reason not to use CUDA.

So the problem is programmability

I don't agree that because the primary supplier is a monopoly is justification for doing something as distracting and risky as designing your own chips.

So they need a second supplier? Do what the government do, and demand that there be a second source https://en.wikipedia.org/wiki/Second_source or you won't buy their product, or give money to some other company or companies for whom that is a core competency.

>> so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer

You say this with great certainty, that almost got me on board, but saying something with certainty does not make it right. It's not critically important - what's the worst that can happen in the short term - they pay a bit more because of the monopoly? You make it sound like if the chip monopoly isn't broken then doom will befall them all. Piffle. If there is such great demand for these chips then there are plenty of other silicon companies who can be attracted by the smell of demand.

What would make sense is for Facebook to claim to be making their own to strengthen their negotiating position with the supplier, but with no real plan to design their own chip.

> ...doing something as distracting and risky as designing your own chips.

This is what you do when your company is a cash cow, you earn so much money that even reinvesting it in the core business doesn't impact your always growing revenue and profits. Also, it is refreshing that new companies are entering the classical semiconductors market, lots of positive externalities.

Why is designing custom chips risky and distracting ? It seems to have worked pretty ok for e.g. google and amazon.

Apple designed its own ARM chips and they design some pretty damn good ones. More competition and choice in the semiconductor space is universally a good thing.

Can you actually purchase an Apple ARM SOC or make your own SOC with their processor for product development? If not, then I wouldn't classify them has a competitor since they only allow for internal usage.

What? Google has their TPUs to do their AI work. On the device they now have their PVC.

https://www.wired.com/2017/04/building-ai-chip-saved-google-... Google's TPU Chip Helped It Avoid Building Dozens of New Data ...

I am not sure many still argue core compute is not gf's core competency. If there are still many, I'd say they did a good job of hiding core competency.

They can ditch CUDA, and start using AMD in addition. It's still not a huge increase in choice, but better than one.

Ditching CUDA would mean accepting at least 10-30% performance hit on NVIDIA cards and losing the ability to use the newest NVIDIA improvements. OpenCL doesn't know about HMMA (aka "Tensor Cores"), so it would be about an order of magnitude slower for DL on Volta.

I think you'd be hard pressed to find people willing trade significant performance for the option of running applications on AMD's less power efficient processors.

Nvidia should provide Vulkan extensions to access those I suppose. So they should still be accessible. There is some plan to converge Vulkan and OpenCL into one coherent combination.

The other factor at play is that Nvidia chips were originally designed for graphics processing tasks, which may not be the best architecture for AI.

... and were completely redesigned starting in the mid-2000s to be good general purpose parallel processors.

Modern GPUs have very little specialized graphics hardware. This doesn't mean there aren't better architectures, but this idea that it's bad cause it's a graphics accelerator is horribly out of date.

A widely propagated myth.

GPUs were largely programmable parallel processors by the time they became in vogue for deep learning.

Bill Dally (nVidia chief scientist) has said that the hard wired graphics takes up so little die area that it is essentially negligible in terms of cost for them.

Plus, the latest Volta chips have "tensor cores" the equivalent of little TPUs.

Reply on news.ycombinator.com