Hacker News Reader

Comment by tambourine_man | Hacker News Reader

> FERRET is trained on 8 A100 GPUs

So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding.

[edit] I meant M series, Apple Silicon

By "A series" are you referring to the Nvidia A100 or the Apple A-series iPhone/iPad chips? If the latter, I don't think you can draw that conclusion. Training has memory and processor requirements that are very different from inference. You don't need iPhones and iPads to train models—you need them to run models. These are two very different things.

Apple Silicon, sorry for the ambiguity. Apple sells Macs too. That’s where I’d hope they would train their models.

I feel like Apple is only testing the waters with AI right now, but perhaps if they get involved enough they'll spend money on their own compute infrastructure? Nvidia is kind of the king at GPU compute right now, and developing comparable hardware is no small or cheap task, but I think Apple is in a very good position to be able to make it work---if they decide to invest in it. But honestly, as far as corporate feud goes, I feel like companies will happily suck it up if it makes some process cheaper and/or easier

I think you’re completely correct, but if they were caught off guard by the AI train, they shouldn’t be testing the waters now. It should be treated as an existential threat.

I've always wondered why Apple is never in the conversation of modern AI, like until now it almost feels like they've been just watching from the sidelines without taking part in all the commotion

Maybe they can just afford to observe before making major decisions on the direction of the company... For a company like Apple, I feel like they won't lose their customer-base just because they are taking the AI race slowly, in fact Apple has often been late to introducing very common feature

I hit reply before I finished typing XD

... Anyways, my point being that Apple will gladly introduce a polished product a couple of years after everyone else has already done it, and their target audience will still applaud their work and give them money. Apple for some reason simply _can_ afford to test the water

You can edit your comments shortly after you make them.

Why would they dogfood Apple Silicon for training models? Seems like a waste of developer time to me.

Apple doesn’t even sell NVidia cards on their Mac Pros. Are they training it on Linux?

I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried.

AI/LLM is the new hot thing. If people are using Windows or Linux, you’re loosing momentum, hearts and minds… and sales, obviously.

But no one is training these kinds of models on their personal device. You need compute clusters for that. And they will probably run Linux. I'd be surprised if Microsoft trains their large models in anything else than Linux clusters.

Apple used to sell servers. I don’t thing they should settle for “just use Linux” in such and important field.

Why does the OS matter for training models?

Apple would want to train models as fast as they could. Nvidia provides an off the shelf solution they can just buy and use for a very reasonable price and sell on the second hand market.

If they wanted to use their own hardware they would either need more of it, which would cost a lot and divert production from sellable devices; or they would need to make special chips with much bigger neural engines, which would cost even more.

Also Apple uses public clouds for service stuff. They may not even own any hardware and just be renting it from AWS/Azure/GCP for training.

I feel like I’ve answered the issues you raised in this thread already.

> Used to

Exactly, over a decade ago...

> But no one is training these kinds of models on their personal device

on-device transfer learning/fine tuning is def a thing for privacy and data federation reasons. Part of the reason why model distillation was so hot a few years ago.

If a train with a GM diesel engine delivers raw materials to a Ford factory for making F150s, you would conclude that consumers whould start driving trains to work?

Is that your argument?

Not at all, just that the engineers at Ford would be more proud if the train used their own diesel engine. And that this kind of thing affects public perception. “Ford is not for heavy duty”

Don't think that follows. I doubt all their cloud services run on Apple hardware. They make consumer devices.

AFAIK all their cloud services run on x86 hardware with Linux, including Xcode Cloud (which runs macOS in a QEMU VM, in a way that only Apple can technically and legally do).

It has always been x86 Linux going back to the NeXT days.

Today they have large scale Kubernetes ands some legacy Mesos clusters.

This

They ceded the data center environment years ago.

Apple doesn't even support NVidia cards on their Mac Pros. The technical details are above my head, but the way Apple M* chips handle PCIe make them incompatible with GPUs and other accelerator cards. Whether you use macOS or Linux.

Apple doesn't even make rack-mount server hardware. It's not that surprising.

Apple makes very capable, efficient devices for end users and content producers. End users do not normally need to train new models.

They do make rack mount hardware:

https://www.apple.com/shop/buy-mac/mac-pro/rack

Don't know if that qualifies as server :).

They used to sell the best developer’s machines. Before Docker, at least. Don’t developers need to train models? And if they are doing so on Linux or Windows, wouldn’t that make it easier for them to better or exclusively support the end product on the one they’re more used to?

You want to be the platform where the excitement is happening.

> Are they training it on Linux

Yes. Apple has always run their servers on Linux e.g. App Store, iTunes.

And training isn't right now an end user activity but something reserved for server farms.

> I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried.

What percent of Apple's customers train models? Does it even crack 1%?

Apple already fails for many types of computing, e.g. any workflow that requires Windows or AAA gaming.

Right, and they are trying to attract games and gamers. “Macs are bad at gaming” shouldn’t be something they settle with.

Apple had a falling out with Nvidia a number of years ago. I believe they were using chips from Nvidia in their MacBook Pros, the first to come with both integrated and discrete graphics, but the solder between the chips and the motherboard kept cracking and a rather large number of MacBooks needed to be repaired.

https://techcrunch.com/2008/12/09/scientists-nvidia-put-faul...

As long as the inference can be done locally on their chips I don't at all think it's a big deal to train models on Nvidia/other hardware.

Are all the iCloud servers running on Apple silicon? I assumed they were running on standard rack mounted hardware.

I think Apple considers cloud infrastructure a necessary evil and a commodity.

AI isn’t, yet at least, and I don’t think they can afford to treat it as such.

yeah, aren't the new M3 chips supposed to be really good for ML training?

Apple apparently uses Google Cloud, so it's that or TPUs!

They use many clouds. But LLM should be their core business and they usually don’t outsource that.

That's not how s/w or any development work. Even if there is a team working on M4 SuperUltra++ which competes with H100, in meanwhile, s/w team will continue to use what is available which may be Intel/AMD PCs with Nvidia GPUs or GoogleCloud/Azure/AWS.

Sure, but there’s also the possibility that they aren’t developing the SuperUltra++

Reply on news.ycombinator.com