Hacker News Reader

Comment by Nevermark | Hacker News Reader

Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU <--> CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras.

They have been taking NPU's seriously on their phones, tablets and laptops since the M1.

Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's = 2TB RAM. Perfect for a large Mixture-of-Experts model.

It would be very strange if they didn't notice their product line had landed in a new sweet spot.

View on HN

FWIW, the reported reason for OpenAI buying Macs has nothing to do with the memory by the sounds of it. Every single outlet I can find reporting on this seems to repeat that the intended use case is for agentic workloads and generating training data for reinforcement learning. They don't appear to be doing inference nor any sort of training AFAICT.

For OpenAI, their data center archipelago is their own "local" and "personalized" AI.

I am curious about the corporate disconnect from the frontline to the generals.

While the company I am in is embracing AI the disconnect and delay between what is available and possible versus what is approved and permitted is a three month window. The State employees I speak to are just now getting around to writing their usage policies for internal AI usage.

Same here. For every little use of AI we have to fill out a 3 page proposal to get a PoC for 3 weeks in copilot studio (the worst AI builder around but due to MS lobby the others are banned). Then we have to find a business sponsor to decide it's useful, find 2 financial controllers to underwrite its token cost (even if very little), go through rounds of lifecycle and governance approvals. Then we can move to preprod but nooo we're not there yet. Now comes the security review and DPIA. That's another hugely complicated process that takes months. Eventually when both are done we can go the the deployment team and go through their process which I haven't even seen yet because most of our projects got cut off beforehand.

All the while the top of the company is of course praising AI and saying we should do everything with it right away.

It's a joke really. We had to go through all this rigmarole just for an agent that does some preliminaries on a service support ticket (making sure all data was filled in correctly and contacting the requester of not) before sending it to a human for final review. It couldn't action anything, not a single thing. Just assist with the prerequisites.

And yet they call our company 'innovative'. We're certainly innovative at inventing bureaucracy.

Reply on news.ycombinator.com