Facebook Is Forming a Team to Design Its Own Chips | Hacker News Reader

Facebook Is Forming a Team to Design Its Own Chips

bloomberg.com

133 pointsjonbaer58 comments

People are making fun of facebook/google for expanding beyond core competency, but this is a market (AI training / inference) where Nvidia has a very real monopoly that facebook and google must rely on for core competency products, so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer and partnering with TSMC (or whoever) to make training and inference chips. It's absolutely the right decision today, and if there was one viable alternative to Nvidia (which AMD and Intel are not), it wouldn't be necessary.

The entire reason that AMD hardware isn't considered a viable alternative to NVIDIA is that it doesn't run CUDA, which is going to be true of these custom chips too.

The chips FB are working on are probably far more similar to Google's TPUs than anything NVIDIA makes though.

For researchers, yes. At the scale Facebook and Google operate at the lack of CUDA isn't much of a problem. The big difference is that AMD's GPUs only work on vectors whereas Google's TPUs (and NVidia's latest card) can work on matrices directly for a large increase in throughput.

Most DL researchers don't care about CUDA, they use higher level frameworks (Tensorflow, Pytorch, Caffe, etc). The frameworks can be extended to support other backends (e.g. OpenCL). If someone builds their own hardware, provides integrated support for that hardware in Tensorflow, and demonstrates speed advantages, people will use it.

This is something that I've wondered about. There is nothing about Tensorflow (for example) that is specific to CUDA. Why doesn't AMD dedicate a couple of engineers to port it? They don't even have to do a complete job. Only about half of the Tensorflow operations are used for most models.

AMD does. https://github.com/ROCmSoftwarePlatform/tensorflow

If they make CUDA the industry standard by porting it to AMD, and then Nvidia is still able to control the standard, over the long term AMD will always be in the position of having to catch up.

I think the GP is suggesting AMD port TensorFlow to OpenCL to break up the CUDA monopoly.

No need, there is already an industry standard that works across all hardware.

Why do you say that? Because they already have a large code base that would be too much work to move over? Or superior features in CUDA? This is actually related to some ongoing research for me, be very interested in any insight.

Because there's no compelling reason not to use CUDA.

So the problem is programmability

I don't agree that because the primary supplier is a monopoly is justification for doing something as distracting and risky as designing your own chips.

So they need a second supplier? Do what the government do, and demand that there be a second source https://en.wikipedia.org/wiki/Second_source or you won't buy their product, or give money to some other company or companies for whom that is a core competency.

>> so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer

You say this with great certainty, that almost got me on board, but saying something with certainty does not make it right. It's not critically important - what's the worst that can happen in the short term - they pay a bit more because of the monopoly? You make it sound like if the chip monopoly isn't broken then doom will befall them all. Piffle. If there is such great demand for these chips then there are plenty of other silicon companies who can be attracted by the smell of demand.

What would make sense is for Facebook to claim to be making their own to strengthen their negotiating position with the supplier, but with no real plan to design their own chip.

> ...doing something as distracting and risky as designing your own chips.

This is what you do when your company is a cash cow, you earn so much money that even reinvesting it in the core business doesn't impact your always growing revenue and profits. Also, it is refreshing that new companies are entering the classical semiconductors market, lots of positive externalities.

Why is designing custom chips risky and distracting ? It seems to have worked pretty ok for e.g. google and amazon.

Apple designed its own ARM chips and they design some pretty damn good ones. More competition and choice in the semiconductor space is universally a good thing.

Can you actually purchase an Apple ARM SOC or make your own SOC with their processor for product development? If not, then I wouldn't classify them has a competitor since they only allow for internal usage.

What? Google has their TPUs to do their AI work. On the device they now have their PVC.

https://www.wired.com/2017/04/building-ai-chip-saved-google-... Google's TPU Chip Helped It Avoid Building Dozens of New Data ...

I am not sure many still argue core compute is not gf's core competency. If there are still many, I'd say they did a good job of hiding core competency.

They can ditch CUDA, and start using AMD in addition. It's still not a huge increase in choice, but better than one.

Ditching CUDA would mean accepting at least 10-30% performance hit on NVIDIA cards and losing the ability to use the newest NVIDIA improvements. OpenCL doesn't know about HMMA (aka "Tensor Cores"), so it would be about an order of magnitude slower for DL on Volta.

I think you'd be hard pressed to find people willing trade significant performance for the option of running applications on AMD's less power efficient processors.

Nvidia should provide Vulkan extensions to access those I suppose. So they should still be accessible. There is some plan to converge Vulkan and OpenCL into one coherent combination.

The other factor at play is that Nvidia chips were originally designed for graphics processing tasks, which may not be the best architecture for AI.

... and were completely redesigned starting in the mid-2000s to be good general purpose parallel processors.

Modern GPUs have very little specialized graphics hardware. This doesn't mean there aren't better architectures, but this idea that it's bad cause it's a graphics accelerator is horribly out of date.

A widely propagated myth.

GPUs were largely programmable parallel processors by the time they became in vogue for deep learning.

Bill Dally (nVidia chief scientist) has said that the hard wired graphics takes up so little die area that it is essentially negligible in terms of cost for them.

Plus, the latest Volta chips have "tensor cores" the equivalent of little TPUs.

So they are going to design their own chips for the consumer devices (Oculus Go and the speakers)? Those are two very different use cases, heavy 3D and speakers generally are simple. I am not sure Facebook will have sufficient volume of either of those to justify making its own chips. Apple sold 250M iPhones in 2017. Facebook sold less than 1M Oculus Rifts in 2018.

The other place that Facebook could use its own chips is in its datacenter. This would make a little more sense as it is easier to deploy custom chips into a datacenter you fully control. There are theoretically cost and energy savings possible from switching from Xeon D to ARM in the datacenter at Facebook's scale.

I think the point is not to become a chip producer, but to cut a hard point of dependency. My point is easier to understand for software development. Imagine a society outsourcing 100% of its software development. If it has a problem with a provider, it has no other solution than to ask another one. As all providers have similar business logic, the problem may not be solved. Imagine the society outsources only 95% of its software development. It has a lot more power over its providers because it can decide to do conflictual projects in-house. IMHO all societies should keep technical expertise of their key technologies by having some internal developments (software and if possible hardware).

This is likely the FB job listing: https://www.facebook.com/careers/jobs/a0I1H00000LgnqFUAR/ Looks like all of the difficulty of building a hardware startup, while maintaining all of the downside of being part of a large company.

[IIRC, Cisco used the "spin-in" model to great advantage in order to address some of the downside. However, that was 10-20 years ago, and I don't know any recent examples.]

A more recent example for Cisco was Insieme from 2013, which became ACI and Nexus 9k products.

https://www.fiercetelecom.com/telecom/cisco-sharpens-sdn-foc...

It's an interesting timing - Qualcomm just announced major layoffs so it's possible some folks from there will apply.

Yann LeCun posted the ad for this yesterday on FB.

Maybe they are doing Oculus too, but it sounds a lot to me like they are doing custom A.I. related silicon.

There's a well proven path showing that custom silicon can save energy relatively easily on inference ML tasks. Competing with NVidia on training is harder of course, but maybe possible for specific tasks.

Frightening to think of how FB will manage a chip platform given their reckless behavior with our data. I hope it never leaves the whiteboard.

You do realize that FB has been designing their own servers for years, right? Also, what does personal data privacy have to do with silicon design? I know that FB privacy rules are a hot topic right now but how does that have anything to do with this article?

The Facebook of Things, delightful.

They can spend say $5 million in employee time and fpga devkits and know within a few percent what the chip performance is, and they have a huge corpus for testing. Doesn't seem to be a huge financial risk.

This makes perfect sense. It's all about economics both w respect to intel and nvidia. You can't be paying 8-10k per gpu, and you are not going to pay that when you can make something far cheaper and faster. Google got this done in what two years?? All the debates around the tech comparisons miss the economic picture. Who cares if 4tpu's being compared to one volta. Point is those 4 chips together cheaper than buying one v100. Bitcoin mining no different. Do you care how many chips (196) are in an antminer s9? No! What matters is that it mines 13000x faster than a gpu for roughly double the price. Facebook, Amazon and the likes have every incentive to go down this custom ASIC path. Google is now hiring sales folks for their tpu cloud. There is a reason they are not selling the hardware. It's more valuable to them to get tenants the other hyperscale competition can't match. So think all the hyperscale guys now looking for ways to keep up which obviously is horrible for nvidia. A lot of people have missed how much of nvidia's story been about essentially killing it on one very narrow use case, ml training, in a very short time. This is basically 50%+ of all profit growth they have achieved last two years. Nvidia has got some serious challenges ahead.

Would these new chips be based on ARM, RISC-V, or some new ISA?

Hardware-level user tracking? You just know they must be thinking about doing this.

Sorry you are getting down-voted. In fact, Intel and AMD have thousands of undocumented instructions that folks have only started [1] to enumerate. Even then, some of the instructions may not be visible through this method of enumeration. FB would have the option to add their own instructions. I would be curious to see how transparent they might be around the production and documentation of these chips.

[1] https://github.com/xoreaxeaxeax/sandsifter

If you think there aren’t already hardware backdoors inverted by China in most of the chips you use you are naive

Intel ME on steroids ...

Possibly. Copyright laws protect closed source hardware/software everywhere, so anyone putting spyware into a closed source chip would gain one more perfectly legal layer of protection against auditors.

Heartbleed? Get ready for Facegash!

I prefer Clusterzuck.

As in "The handling of Cambridge Analytica was a complete clusterzuck from the start".

I love when companies wander beyond their core business and venture on something they are not familiar! That idea has everything to work it out just fine, don't worry!

Microsoft should stick to basic interpreters.

Apple should only do desktop PCs. Music players, that's ridiculous.

Oracle should only do databases.

Google should stick to search. What do they know about operating systems? They also have no business running their own datacenters, what could they possibly know about allocating ten billion dollars in capital annually to such an operation.

Amazon is a retailer. Cloud services, artificial intelligence, devices, is that a joke?

Intel makes memory chips, what do they know about processors?

Facebook has $41 billion in cash. They're about to start piling up $20 billion per year. Zero debt.

They can do almost anything they want to within reason and not worry about the financial consequences. Their shareholders will be a lot more upset if they don't take pragmatic business risks and pursue opportunity.

"They can do almost anything they want to within reason and not worry about the financial consequences."

No, they need to make the soundest possible investments with that money. Lots of free cash flow is not license to do "anything they want to within reason," especially for a publicly-traded company (sure, if you're talking about a private partnership and all the partners agree, go nuts, I guess). If they don't actually need it to run the business making the profits, they should return it to shareholders since, you know, that's whose money it is.

But there's no point paying dividends to shareholders who aren't demanding them.

Why would a company 'throw away' surplus cash to passive uncomplaining shareholders instead of intercepting it upstream and ploughing it back into tax-reducing activities? Like burning cash on R&D that provides opportunities for pivoting in the future. Now that is prudent and sound.

Eventually shareholders might gain enough voting power to demand dividends but there's no point in piling-up taxable cash until that happens.

$20 million for 14 nm ASIC is not expensive experiment looking at these numbers.

Intel also has been trying to compete with Nvidia, and has completely failed. Processors have been their core competency for a long, long time.

On AI, yes. On graphics cards? Intel graphics have basically eaten the entire bottom half of the market. nVidia doesn't even bother to make anything slower than a xx50 card because it would be slower than the integrated graphics.

nVidia had to diversify to avoid being shoved into a shrinking niche market.

Intel has been integrating the GPU into the CPU for a while now, so that's not really a fair comparison. If you dominate the CPU market, then clearly you can slap a low-end GPU on the die and sell it cheaper than a GTX 1080.

It's a very logical extension of the project that became Open Compute, which started at Facebook back in 2009 and was announced two years later [1].

Facebook has been running its own datacenters, first in colos and now fully purpose-built, for nearly a decade and a half at this point. That hardware design done by a company isn't familiar to you doesn't mean it doesn't exist.

And that's not even mentioning the now nearly four years old Oculus acquisition or the resources Facebook has been pouring into AI/ML research and applications.

[1] https://www.facebook.com/notes/facebook-engineering/building...

To my knowledge, for Open Compute Project Facebook gave specifications to Taiwanese greybox vendors. It's up to the vendors to work out the details of the design, such as component selection and PCB design.

Doing their own ASIC design is a whole other ballgame.

That's correct. They aren't really designing them. Also, making a chip that's as competitive as an NVIDIA card isn't something that you can just make easily. See AMD and Intel.

Phone companies made fun of apple before they made iPhone.

Apple had history making computers - even portable ones - and consumer electronics.

iPhone is just a different form factor in that sense.

This is more like your local sandwich bar deciding to also farm their own pigs for the ham.

If selling sandwhiches brought in 40 million in revenue last year, and there is essentially a hog farmer monopoly (nvidia), and you are worried about the monopoly running out of meat, or selling it first to your competitors, or just hiking prices indefinitely, yeah it makes sense to hire some hands and get yourself a barn.

Reply on news.ycombinator.com