Macs to Get AI-Focused M4 Chips Starting in Late 2024
macrumors.com
macrumors.com
This is not speculative. Almost all of this stuff is already shown to work, and it is all improving incredibly quickly as we speak! It just needs to be assembled into an actual product, which is exactly what Apple does best. With their silicon advantage they are best positioned to ship these kinds of features, and the best part is it can all run offline locally. No data sent to servers. No privacy concerns. Zero network latency.
Perhaps, but a 1TB RAM upgrade would probably cost about as much as a new car. This is Apple we're talking about.
Apple's Lisa was about $30,000 (today's dollars)
I look forward to checking the "+$1000" box for a terabyte of on-chip RAM
$4,859.99 for 8 128GB LR-DIMMS on amazon: https://www.amazon.com/8x128GB-DDR4-3200-PC4-25600-NEMIX-RAM...
The new Mac Pro (with the M2 Ultra) uses unified memory, so I guess no more third party upgrades? Looking here:
https://everymac.com/actual-maximum-mac-ram/actual-maximum-m...
The new Mac Pro is limited to 192GB while the previous Mac Pro can go up to 1.5TB.
Seriously, though, RAM should be getting cheaper/larger over time.
I could do a SFF PC or a Mac mini. After looking at the paltry memory/storage Apple chose to give with the mini, I ended up going with an AMD system with 96GB of ram and 4TB of SSD storage for less than a 16GB Mac mini.
LOL. Have you seem the pricing of Mac memory?
Of course, I can imagine that there will be a significant discount even just considering the price of high end branded memory outside Apple.
Still, Apple has showed no sign of abandoning 8GB for regular people. Even if they switch by the end of the year on their high model, they have a serious handicap in their own installed base.
Apple wanted $3,000 for 160GB of memory for my Mac Pro (i.e. going from a base 32 to 192GB).
OWC sells the same memory (same spec, same manufacturer) now for $620, though I believe I paid about $1,000.
They also wanted another $3,000 for an 8TB SSD. Similarly, I was able to buy a 4xM.2 PCI card, and populate it with 4 x 2TB SSDs for under $1,500. Furthermore my PCI setup is FASTER than the Apple drive (7GBps versus 5.5).
No thanks, I like that my computer only speaks an incredibly specific language which I can use to tell it exactly what to do, and know it’s not going to do anything else. If I wanted a PA, I would hire one.
The "no data sent to servers" point I agree is super important for work things. I pay for and use Github Copilot for personal things but I'm not allowed to use it at work for obvious reasons. If we could remove the privacy concerns and run these models on local hardware (that people actually have) then it'd be game changing.
Reading some of the other anti-AI comments is a bit odd. I guess it's going to depend on what you do for work, but as someone who writes software not using Copilot is like comparing myself from a 0.25x developer to a 2x developer. I mean that when I say it, but I get that you do need to be able to identify wrong code faster than you can run it.
https://www.gartner.com/en/research/methodologies/gartner-hy...
We have not reached the "Peak of Inflated Expectations" yet in AI. Contrast that to Blockchain and Cryptocurrency, where we are definitely in the "Trough of Disillusionment".
Bitcoin has been hovering around its all-time high of around $74,000 and is still expected to continue to increase, especially now that several ETFs have been approved.
There is an aspect of personal bias for sure: I don’t want my profession or the countless related fields to be devalued, I dislike the imprecision of LLMs, and they spark none of the joy in me that programming does.
But there’s something beyond that I find difficult to put into words. There is a tremendous inelegance to it all, in the same way a brute force solution feels fundamentally unsatisfying.
Most software today only uses a tiny fraction of the capabilities of the hardware, because we optimise for how fast we can implement marketable (and often hostile) features. LLMs and co strike me as dialling that up to 11; why improve anything when we can just throw more hardware at the stuff that sucks?
I like browsing the web god damn it, and I don’t need some corporate facsimile of a human to ask me why I don’t look happy.
I actually think local LMs are the antidote for this. Every UI I use these days is optimized by designers for attractiveness and hordes of PMs for business KPIs, not engineers for useful functionality. With local LMs I can take back control over the UI I actually use. The LM can deal with whatever trash UI redesign is being forced on me, and present a much simpler UI that matches my specific needs.
For example, it can summarize the news and I never have to visit a news site and be assaulted by dark patterns. It can collect family photos from Facebook and show me a few without random TikTok videos inserted between them. It can trawl Amazon and Target and Walmart to find a shortlist of products for me to pick from, filtering the trash. It can watch that 10 minute long YouTube video past the sponsored segment to find the nugget of information I need to fix my dishwasher or whatever.
We're a long way from this now. But I see a path there, and a world where users have control over the UIs they use is a better world.
I would love to use a third-party Instagram app that only shows posts from accounts I follow, but I can’t because Facebook has locked that down. I can just as easily see a future where I can summarise my feed, but only if I use the Metabot or whatever which is just implementing some different dark pattern.
Although that does bring me back somewhat to my initial point in that it all feels like brute forcing our way out of problems instead of solving them. We could have a Facebook that shows a nice summary of your family photos, we could have a shopping search engine that doesn’t include the trash, we could have short and to the point videos that aren’t optimising for the engagement flavour of the month.
We don’t because there are too many financial incentives pushing back and too many established players that can burn piles of money for much longer than you, and I just do not see how those same facets won’t just be translated for models.
The point is that this is a local model. You don't have to use Meta's AI because Meta can't stop you from running a model on your own computer to browse Instagram for you and filter it however you like.
I'm sorry to be so pessimistic about it, and I genuinely do hope we see some positives out of this stuff. But I'm becoming increasingly disillusioned with what so much of the software industry has become, and seeing how much of the AI space is led or backed by the tech giants, I can't help but assume the worst.
No it won't. Apart from the fact that (if the article is correct), M4 will support maximum of 512 GB (and I don't want to know how many thousands of dollars you will have to spend on it given how much Apple wants you to pay for a difference between 8 GB and 16 GB MacBooks...), GPT-4-level models that you could run locally don't exist and it would be a challenge in itself to create them. The closest, Claude 3 Opus, as far as I can tell can't be run locally, and it is in nobody's interest to change this status quo (less capable models = run locally, powerful ones = pay us).
I do hope this changes one day, but it is not directly related to M4 (or M5/M6).
Earlier this year I popped for a M2 Pro 32G. I am amazed at what even that can run, but I have to use Ollama to run individual models for general text processing and NLP, one for medical advice, one vision model, one large uncensored model to get an understanding of general utility of censored vs. uncensored models (I wish all models I used were uncensored and trained just on English. I have only used non-English language support for a handful of interactions).
I also want Apple to develop very strong support for cloud models for my Apple Watch. With the excitement and disappointment for the new AI alternative pin computers, etc., I say we already have something useful in the Apple Watch so just keep improving that. I like to go about my day phone-free and the Apple Watch makes that work out OK for me.
Are there any threads left on HN that can be held on topic and not be invaded by the same ol' beaten to death "muh Apple M1 so good, PCs so bad" trope? It's been 4 years already since the M series launched, so unless people have been living under a rock, everyone already knows what they can do, what exactly were you bringing on this topic with your pointless shopping anecdote that has nothing to do with the topic?
And FYI, not everyone is a web dev. Many many jobs will require you to use tools exclusive to Windows or Linux hence why the market for expensive PC laptops is still very large despite being worse on paper than MacBooks.
MacOS doesn't solve everyone's needs no matter how good their chips are, if the SW you need doesn't run on it then it's useless, so Apple is no thereat to that market.
More than you might think. You never know who's reading.
You must live in some coo-coo land of privileged entitlement if you think that not using Apple M laptops somehow equivalate to "suffering". Go out and get some perspective on what suffering actually feels like. I'm ending my talk with you here since your comments are of even lower quality.
Usually for me the culprit for poor battery life is MS office apps, but under MacOS I've seen window manager consume 90% CPU regularly.
Long story short, buggy SW dictates battery life on modern laptops, with very little between apple and zen (can't speak to modern Intel CPUs, i dont have a laptop with one)
So yes, software is the culprit but the M-series are much better at managing the same software (and even heavier ones) that I run, for the same job.
So my point was, and in agreement with you, is when using similar sw (buggy or not) a modern zen4 will have similar battery life to a M1 / M2
For AI, unified memory is _way_ cheaper than high-ram GPUs.
I do think it would be cool to have an M_x chip to try and run ML stuff on.
This makes total sense but does it? Does Apple really have an application roadmap to ultimately utilize these tensor cores?
For all the M1, M2, M3s out there, Siri still sucks. We keep seeing latest arXiv papers [1] coming out of their research efforts hinting at potential improvements but there is not much in those papers that needs to wait for an M4 or M5...
What gives?
https://arxiv.org/abs/2404.05719
Siri is about to get swole.
IIRC their Pixel 8 chips are so underpowered that they can't even run Google's smallest LLM models locally on device, so despite their whole AI hype sales pitch, everything on the Pixel 8 is still run in the cloud.
How is Google so slow at this? At least the likes of Intel and Microsoft have an excuse, they're old and crusty corporations that move slow.
Sure, AI features will come to all of their content creation apps at some point, most of it being processed on-device.
But I suspect the most impact will be in other apps and areas of the operating systems that typically haven't gotten a lot of attention traditionally.
I'd much rather have generative AI or an intelligent agent to deal with email than filling in backgrounds in images. Because the pertinent information is in my calendar, address book, etc., I should be able to say "email Tom I can't make it to the party" and that email is created for me.
Apple has a fairly severe case of Apple Maps-related PTSD.
They aren't likely to introduce a half-baked product in a rush to market like everyone else is doing.
Apple is unlikely to release a fundamental modern-ML update to Siri without it both breaking new ground in features and being considerably more "safe" to use than other products ("safe" in the sense of keeping Apple out of the headlines).
Agreed. That said, if you do AI/LLMs, few, if any, portable non-Mac systems have the ability to pull this off. The Macs just crush it when it comes to GPU and memory bandwidth performance, and they do this while sipping battery life.
Sure, most of that is possible server-side – but the appeal would be that local processing could preserve a lot more privacy.
You give. More money.
These new models aren’t going to sell themselves.
Question: Is it illegal for apple to do something on-device that runs queries against all the databases for the user's applications?
Why would it be illegal?
I am not an Apple user though, so I have very little experience with Siri.
Apple always does things in a holistic way. They wouldn't put these AI cores in their devices if they didn't also have a software strategy to go along with it.
Unlike with most companies, AI won't be bolted on; it'll well-integrated into the operating system.
For example, they've been working on LLMs[1]:
> We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.
And it's not going to require a terabyte of RAM either.
You can read the details: "ReALM: Reference Resolution As Language Modeling" [2]
There's also evidence of an AI browsing assistant for Safari. [3]
[1]: "Apple AI researchers boast useful on-device model that 'substantially outperforms' GPT-4"--https://9to5mac.com/2024/04/01/apple-ai-gpt-4/
[2]: https://arxiv.org/pdf/2403.20329.pdf
[3]: https://www.macrumors.com/2024/04/10/ios-18-safari-browsing-...
(Aside: Apple's MLX framework is showing nice performance gains over PyTorch on their silicon. I'm excited to see that project evolve.)
You mean it's silly how far ahead Apple is since they offer 192 GB of VRAM while Nvidia only allows 24 GB for reasonable prices? Or do you mean it's silly to compare <$10K Macs with >$30K Nvidia setups in the first place?
I'm not an Apple fanboy - I just think Apple as a company has been prepared and thinking about this stuff for literally decades. They set the bar for user friendly products. It won't be the first time that Apple is late to the market, but they've always redefined it when they arrive, becoming the new standard. It's what they do.
I'm a firm believer in AI at the edge: Low latency, privacy, personalization, device integration and no need to invest in massive AI server farms. It's in Apple's best interest to bring AI computation to the masses.
But we'll see if Tim can pull it off.
Just a hunch.
A sudden exponential set of improvements that allowed everyone to dream that self-driving anything was around the corner. Actual promising real world result that show we are really no that far.
But the last bridge to cross is actually extremely slow and what was around the corner, becomes a decade away.
That said, it's not fruitless. AI is useful as it is today, and if does not go pick up your kids at school in the next 10 years, it would still be useful.