Apple approves driver that lets Nvidia eGPUs work with Arm Macs
twitter.com
twitter.com
You want to use an NVidia GPU for LLM ? just buy a basic PC on second hand (the GPU is the primary cost anyway), you want to use Mac for good amount of VRAM ? Buy a Mac.
With this proposed solution you have an half-backed system, the GPU is limited by the Thunderbolt port and you don’t have access to all of NVidia tool and library, and on other hand you have a system who doesn’t have the integration of native solution like MLX and a risk of breakage in future macOS update.
The software stack has been ready for Apple Silicon for more than a half decade.
> the hardware wasn't usable on macOS
This eGPU thing is from a third-party if I understand correctly. I don't see why nvidia would get excited about that. If they cared about the platform, they would have released something already.
Even on Intel Macs using eGPU with nvidia cards was near impossible. nvidia just doesn't care about it after the breakdown of the two companies' relationship.
Whether a third party has created a signed driver or not doesn't matter much until there is more interest from the GPU maker. This barely moves the needle.
If a model can run on a 512GB M3 Ultra via MLX or CUDA, but simultaneously benefit from the memory bandwidth of something like an RTX 6000 Pro; that would save my company hundreds of thousands of dollars. That's $20,000 for roughly 600GB of VRAM, and enough token generation speed to fulfill the needs of any enterprise that's not a hyperscaler or neocloud.
I'll let someone else do the math for you on what it costs to put together a 10U server to get that kind of performance without the $10K M3 Ultra Studio.
What we're paying for five old 80GB A100s is criminal, but it's nothing compared to what these GB200 Blackwell setups are going to cost in 2030. Market economics aside, the fact that they require sophisticated liquid cooling infrastructure and draw 3x the power of the A100s, will make these cards unattainable for small to medium organizations.
So yeah, if there's some outside chance that we can pair NVIDIA's speed with a an arm-powered machine that offers 512GB Unified Memory while drawing 50W -- you better believe it's a big deal. We'll see. Sounds too good to be true.
Yes, for many scenarios this is "not even an academic exercise".
For a very select few applications this is Gold. Finally serious linear algebra crunch for the taking. (Without custom GPU tapeout.)
Not everything is limited by the transfer speed to/from the GPU. LLM inference, for example.
I thought Thunderbolt was like pluggable PCI? The whole point was not to limit peripherals.
he actually did reply weeks later and said "i didnt realize people wanted this, my team has added them. go check now". pretty sure that was the last time nvidia drivers came to macos.
there's a lot of assumptions made with this topic, particularly the assumption that apple is blocking them. at least in my experience the opposite was true, nvidia just flat out wasn't making them. however i don't doubt the truth lies somewhere in between: nvidia and apple have a pretty much nonexistant relationship now. i dont know whats required here but i also don't doubt apple makes this experience suck butt for any interested parties.
nvidia employees: please fwd!
Before that was the pre-trash can Mac Pro in 2006-2012. So that was canceled most of a decade before the 2019 model.
High bandwidth PCIe hasn’t been a thing in Apple world for most of 15 years.
Also Thunderbolt is trivially disconnected, which in many critical workflows is not a positive, but an opportunity for ill-timed interruptions. Plus I don't have to buy a fucking dongle/dock for a real goddamn slot, make room for external power supplies, etc.
Apple has a monopoly over the "M-chip" personal computer market. They have a monopoly over the iOS market with the app store. They have a monopoly over the driver market on macOS.
Like, Microsoft was found guilty of exploiting its monopoly for installing IE by default while still allowing other browser engines. On iOS, apple bundles safari by default and doesn't allow other browser engines.
If we apply the same standard that found MS a monopoly in the past, then Apple is obviously a monopoly, so at the very least I think it's fair to say that reasonable people can disagree about whether Apple is a monopoly or not.
[0]: https://en.wikipedia.org/wiki/United_States_v._Microsoft_Cor....
The relevant thing here isn't the chips, it's tying things to the chips, because those would otherwise be separate markets. If you could feasibly buy an iPhone and install Android or Lineage OS on it or use Google Play or F-Droid on iOS then no one would be saying that Apple has a monopoly on operating systems or app stores for iOS since there would actually be alternatives to theirs.
The fake alternative is that you could use a different store by buying a different phone, but this is like saying that if Toyota is the only one who can change the brake pads on a Toyota and Ford is the only one who can change the brake pads on a Ford then there is competition for "brake pads" because when your Toyota needs new brake pads you can just buy a Ford vehicle. It's obvious why this is different than anyone being able to buy third party brake pads for your Toyota from Autozone, right?
> It’s also probably relevant that MS was not selling PCs or their own hardware.
This is the thing that unambiguously should never be relevant. It can't be a real thing that you can avoid being a monopoly by owning more of the supply chain. It's like saying that Microsoft could have avoided being a monopoly by buying Intel and AMD, or buying one of them and then exterminating the other by refusing to put Windows on it. That's a preposterous perverse incentive.
Move the most important aspects of your software to hardware. Hard for MacOS but for a Chromebook style thing you could write the browser into its own pice of wafer.
Google should pay me to be this evil.
So now you have a piece of silicon with a two year old version of Chrome with seventeen CVEs hard-coded into it, and still have all the same antitrust problems because the device still also has an ordinary general purpose CPU that you're still anti-competitively impeding people from using to run Firefox or Ladybird.
Microsoft was found guilty, so clearly the bar is not what you're trying to claim.
Go ahead, I'll wait.
There are plenty of Linux distributions that use immutable root volumes. They protect the user in a huge number of ways by preventing the system from getting hosed (either by accident or by malicious unauthorized users / malware). Apple made the decision to do this for their users, and it has prevented a HUGE amount of tech support calls, as well as led to millions of happy users with trouble-free computers.
It also hasn't stopped users from installing Chrome and/or Firefox on their Macs, and millions of ordinary users have.
You seem to be ignoring the part where you can't install the Chome and/or Firefox browser engines on iOS and the apps with those names on that platform are just skins over Safari. Notice in particular that the iOS version of "Firefox" can't support extensions.
> Let me know how I can unbundle Safari from macOS or iOS.
> Go ahead, I'll wait.
You can't get even macOS from the store without Safari, which is the thing Microsoft was doing, but what Apple does on iOS is far worse than what Microsoft was doing and talking about only macOS is kind of burying the lede.
What next? Do you want to unbundle the built in drivers?
The inability to remove it is just the dodge Microsoft attempted to use to claim that they're inseparably the same product, and was clearly a load of self-serving nonsense. Operating systems had system dialogs before there was any such things as browser engines.
The dynamic looks weird from the frame of reference of the modern browser market because the answer the market found to Microsoft's tying was to "pay for" the browser by allowing the vendor to choose the default search engine. No surprise then that the browser that ultimately supplanted Microsoft's was the one from the biggest search engine company. But that workaround came with negative consequences, e.g. Google now crippling ad blockers in Chrome.
And the tying problem is still there even if markets with low marginal costs are often weird. Okay, so the way we pay for browsers now is by letting the vendor choose the default search engine, but now we have Google paying Apple billions of dollars to be the default search engine in Safari, and Apple quashing Firefox ad blockers on iOS, instead of that money going to Mozilla or Ladybird or anyone else who has to compete by making a better browser instead of "competing" by tying use of their browser to an operating system, with correspondingly fewer resources and market share for competing alternatives.
> What next? Do you want to unbundle the built in drivers?
Making Asahi Linux get there by full reverse engineering actually is kind of a dick move? Intel publishes hardware documentation.
And it seems pretty obvious that Apple is tying their OS to their hardware and vice versa. Is that even supposed to be ambiguous?
> Notice that they quite successfully bankrupted Netscape with this.
Were you around back then? Absolutely no one paid for Netscape even before IE. And famously what bankrupted Netscape was because it “did things you should never do”.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Netscape was trying to make money selling web servers also. Should Linux and Windows not come with web servers? Should Apache not have been free?
People seem to forget that Netscape sucked around the time IE came out. It was so crash prone on every operating system it ran on that people use to brag on .advocacy groups about how good their operating systems were by how well they handled Navigator crashes.
And there has never been a point that Microsoft had to unbundle their browser in the US and there was never a browser choice screen.
> And it seems pretty obvious that Apple is tying their OS to their hardware and vice versa. Is that even supposed to be ambiguous?
This is about as bad of an argument as saying that Fors ties its motor to its cars or Nintendo forcing you to use their OS with their consoles. Apple doesn’t sell operating system, Apple sells computer products. What do you think should happen? Force Apple to create versions of its operating systems that run on other computers? Force Apple to sell Macs without operating systems? Anyone is free to choose an x86 PC and 90% of the market does
Firefox is also free to bundle an ad blocker with Firefox even if it does use WebKit and when you download Firefox for iOS - they get money from searches.
Are you suggesting that iOS shouldn’t come with a browser? Should ChromeOS also not come with a browser?
Absolutely no computer operating system comes bundled with Chrome besides ChromeOS yet Chrome still has the majority of the market share on desktop computers. Firefox competes with Chrome on an equal playing field on computers - people choose Chrome
How about, sell the product without restricting retailers from replacing the vendor's browser with another one, or give the customer a choice which browser they want the same as they choose how much RAM they want etc.
> And famously what bankrupted Netscape was because it “did things you should never do”.
Internet Explorer was bundled with Windows 95. The Netscape release before they attempted to rewrite was released in 1997. The rewrite was a failed attempt to make their browser good enough that people would pay for it when Microsoft was already bundling IE with Windows.
> And there has never been a point that Microsoft had to unbundle their browser in the US and there was never a browser choice screen.
Indeed, Microsoft successfully paid off the Bush administration to settle the case for a slap on the wrist after they'd already been found guilty by the court.
> This is about as bad of an argument as saying that Fors ties its motor to its cars or Nintendo forcing you to use their OS with their consoles.
Ford will happily sell you a motor without an entire car, or a frame or any other part of the car without a motor. Nintendo is forcing you to use their OS with their consoles.
> Force Apple to create versions of its operating systems that run on other computers?
This makes it sound like it's someone making Apple do something instead of Apple making someone do something.
What stops you from running macOS in qemu or a virtual machine on any non-Apple hardware with the same architecture? What stops Samsung from writing iOS drivers and offering iOS on Galaxy phones? Only Apple's refusal to sell it to you without making you also buy hardware.
> Anyone is free to choose an x86 PC and 90% of the market does
60% of phones in the US are iOS.
> Firefox is also free to bundle an ad blocker with Firefox even if it does use WebKit and when you download Firefox for iOS - they get money from searches.
The Firefox ad blockers are extensions, e.g. uBlock isn't from Mozilla, but the ability to use it is a reason to use Firefox. The iOS browsers can't use extensions. Then you can't use uBlock on iOS and fewer people use Firefox.
> Absolutely no computer operating system comes bundled with Chrome besides ChromeOS
Android. And then people who want to use the same browser on desktop and mobile for sync.
> yet Chrome still has the majority of the market share on desktop computers. Firefox competes with Chrome on an equal playing field on computers - people choose Chrome
Chrome is made by the largest advertising company in the world. For years if you opened google.com, gmail or their other services in a non-Chrome browser you would get a huge banner imploring you to install Chrome. This was a successful strategy to overcome the inertia of the default browser on desktop operating systems, but Mozilla never had anything like that available to them, and then the two-front assault from Microsoft/Apple on one side and Google on the other resulted in declining Firefox market share and correspondingly declining revenue with which to improve it.
Mozilla the organization also suffers from significant mismanagement, but that doesn't explain why no one has been able to establish a popular fork or new independent browser, whereas the OS vendors successfully impeding anyone who can't command the equivalent of billions in advertising explains it really well.
PC vendors have been and do ship any type of crapware they want on their computers.
> or give the customer a choice which browser they want the same as they choose how much RAM they want etc. browser they want the same as they choose how much RAM they want etc.
And when they had that choice in Europe - they mostly still chose Chrome…
https://arstechnica.com/information-technology/2014/12/windo...
> Android. And then people who want to use the same browser on desktop and mobile for sync.
And those people can still download Firefox on iOS or Android and sync bookmarks.
In fact Firefox and Chrome Windows users can sync their bookmarks to iOS Safari using extension written and supported by Apple.
> What stops you from running macOS in qemu or a virtual machine on any non-Apple hardware with the same architecture? What stops Samsung from writing iOS drivers and offering iOS on Galaxy phones? Only Apple's refusal to sell it to you without making you also buy hardware.
Is that really a reasonable argument when Samsung doesn’t even support its own hardware with drivers for more than a couple of years?
> whereas the OS vendors successfully impeding anyone who can't command the equivalent of billions in advertising explains it really well.
Just maybe Firefox - which is free to compete with Google on desktop computers just doesn’t make a compelling case for why no one wants it?
But hey, maybe some weird shit happened during the clone years that I’m not privy to.
Just an example… and yes, I know the EU ruling but it’s still fitting.
Compare the games console market. Nintendo is allowed to say you have to go through them to sell games for the Switch, ditto Microsoft with the Xbox. Sony doing the same thing with the Playstation is exactly equivalent, but they're approaching the sort of market dominance where it might soon be illegal for them (and them alone) to do that in some markets.
Copyright (e.g. over iOS) and patent (e.g. over iPhone hardware) are explicitly government-granted monopolies. Having that monopoly is allowed on purpose, but that isn't the same as it not existing, and having a government-granted monopoly and leveraging into another market are two quite distinct things.
> Compare the games console market.
Okay, all of the consoles that require you to sell you to sell through their stores shouldn't be able to do that either.
> but they're approaching the sort of market dominance where it might soon be illegal for them (and them alone) to do that in some markets.
Wait, your theory is that a console with ~50% market share has market dominance but Apple with ~60% of US phones doesn't?
Products and markets are not a one to one mapping. For example, if you sell low-background steel, that's part of the broader "steel" market because anyone who needs ordinary steel could buy it from you and use it for the same purposes as ordinary steel. But low-background steel is also its own market, because the people who need that can't use ordinary steel. Likewise for sellers of products with higher purity levels, products that satisfy particular standards or regulatory requirements, etc. It's only the same market if it's the same thing. Clorox bleach is the same as other bleach; Microsoft Windows is not the same as MacOS.
And iOS is not the same as Android. I mean this really isn't that hard: Are they substitutes for each other? If you have a GE washing machine, can you use any brand of bleach? You can, so they're in the same market. If you have an app that exists for iOS and not Android, can you use an Android device? No, so they're not in the same market. Likewise, if you've written a mobile app and need to distribute it to your customers who have iOS devices, can you use Google Play? Again no, which is what makes them different markets. They're not substitutes, any more than a retailer in Texas is a substitute for a retailer in California when you have customers in both states -- or only have customers in California.
The monopoly that Microsoft held was the home computer operating system market, first through DOS, then later through Windows. Holding a monopoly like that isn't illegal unto itself. What they were actually found guilty of was unfairly leveraging their monopoly on the OS market to gain the upper hand in a different market (the browser market). The subsequent range of issues we had with IE6 (compatibility, security, etc) was a result of Microsoft succeeding in achieving a monopoly on the browser market through illicit means.
Likewise, "Apple has a monopoly on the App Store" is just the same amount of nonsense. What you could argue is that Apple has a monopoly on the home computer market, or the mobile phone market, and that the way they integrate the App Store should be considered illegal leveraging of that monopoly, but that argument simply doesn't hold water — Microsoft's monopoly on the OS market at the time was pretty much incontrovertible, you simply couldn't walk into a shop and buy a computer running something else (except maybe a Mac at a more specialised place). Today, just about any shop you walk into that sells computers will probably have devices for sale running three different OSes (macOS, Windows, ChromeOS). Any phone place will have iPhones and Android devices, and probably a few more niche options. Actual market share percentage is nowhere near the high 90s that Microsoft saw in its heyday. At most, Apple is the biggest individual competitor in the market, but I don't think it hold an outright majority in any specific product class.
Mind you, I think that there is a good argument to be made that the Apple/Google duopoly on mobile devices does deserve scrutiny, but that's a very different kettle of fish.
When a company is deemed an illegal monopoly, the DoJ basically becomes part of management. Antitrust settlements focus on germane elements, e.g. spin offs. But they also frequently include random terms of political convenience.
I don’t think we want a precedent where companies having a product means they have an automatic monopoly on said product.
But the M series are an Apple product line designed by Apple with a ARM license and produced on contract by TSMC for use in other Apple products.
Don’t assume the facts from another case automatically apply in other cases.
Or as Justice Jackson once put it: “Other cases presenting different allegations and different records may lead to different conclusions”
Intel sold chips to anyone. Anyone could make Intel computers.
Apple does not sell chips to anyone. Nobody else can make m-series computers.
Your argument is basically that Ford has a monopoly on selling mustangs because standard oil had a monopoly on selling oil.
lmao what ? the "M-chip" is literally their chip that they designed, built relationships with TSMC over and bankrolled into production to put in their products. literally hardware by apple for apple. this was a decade plus long thing in the making, this is the risk/gamble apple took and invested heavily into. that is apples innovation. any other manuf is free to go do this themselves for their own devices, they just didn't and for the most part still don't. that just like isn't a monopoly at all, i'm amused you even got to that point in the first place. seems to carry some broad misunderstandings of what the M-series chips are or carries an assumption that cpus are supposed to be shared to any interested parties just because that was intels business model. intel was historically slacking & their one-size-fits-most approach wasn't meeting the engineering requirements apple was after generation after generation, so apple took the cpu destiny into their own hands and made their own. if you feel like non-apple laptop chips aren't living up to that kind of perf/ppu.... well yeah you'd be right. but that's not really apples fault. that's not a monopoly thing, like at all. either laptop manufs need to go make their own chip (unlikely) or intel/qualcomm/etc need to catch up.
If we have a right to repair (we broadly do not, AFAICT), then that doesn't necessarily mean that we have a right to modify and/or add new functionality.
When I repair a widget that has become broken, I merely return it to its previous non-broken state. I might also decide to upgrade it in some capacity as part of this repair process, but the act of repairing doesn't imply upgrades. At all.
> No OS provider should be allowed to dictate what software you can or not run on your own device and / or OS you have paid for.
I agree completely, but here we are anyway. We've been here for quite some time.
Apple's decision is not constrained by server logic or ballooning costs, it is entirely a client-based policy to not sign CUDA drivers.
Microsoft rewrote their Windows Phone native client to pass through Google's ads. Google still blocked it.
Was it normal behavior when Google blocked Amazon Fire devices from connecting to YouTube with a web browser during the Google/Amazon corporate spat?
To be fair, Google did back down almost immediately when the tech press picked up on it.
Not allowing a native client for your monopoly market share video service on Amazon devices while also blocking Amazon's web browser on those devices is making things a bit too obvious.
Clients are not offered at-will, they either work or they don't. Nvidia ships AArch64 UNIX drivers, Apple is the one that neglects their UNIX clients.
Google used YouTube as a weapon against both Windows Phone and devices running Amazon's Fire fork of Android.
A "monopoly" "service"? What have they monopolized, laziness? It's not the App Store, you can go replace it with DailyMotion at your earliest convenience.
You're still retreading why your original comment was not at all relevant to the critique being made. We have precedent for prosecuting monopolistic behavior in America, but it doesn't encompass services even when they're mandatory to use the client. It does have a precedent for arbitrarily preventing competitors from shipping a runtime that competes with the default OS, incidentally.
https://www.ftc.gov/advice-guidance/competition-guidance/gui...
have a product with a monopoly market share
AND
use that product as a weapon against competitors in other markets
That conduct is clearly illegal.
But exactly how does Apple have a monopoly in computers with less than 10-15% market share?
Apple doesn't have what American law sees as a monopoly market share in any market.
Be aware that other jurisdictions, like the EU, start placing restrictions on the behavior of companies with lower market share than American antitrust law requires.
If you think otherwise, make your case to Google's lawyers instead of spinning hypothetical case law.
You do own the client though. In the example upstream, the failure to support macOS clients can't be blamed on Nvidia because they already wrote AArch64 UNIX support.
And your example is pretty poor, HBO doesn't have a 10th of the power of YouTube.
The political appointees (of both parties) shut that lawsuit down.
This is as basic as antitrust law gets.
Because of that, you need an apple device around to be able to deal with iMessage users.
In my bubble literally noone uses iMessage. More tech savvy use Signal/GroupMe, less tech savvy use SMS/Email. Family use Signal to chat with me, as I can steer my own family a little.
Also I sometimes open web-interface of Facebook, but any attempts to offer WhatsApp I answer "sorry no Facebook apps on my phone, no Instagram/Messenger either". Never had any issues with that. Although I heard some countries are very dependent on Facebook, so might be hard there.
By the way, I noticed it's not hard to use multiple messengers actually, sometimes it's even faster to find a contact as you always remember what app to look at in recents.
UPDATE: My point is that you can also influence your life and how people communicate with you. Up to a point of course, but it's not like you can do nothing with it.
Your green bubble? =P
My social circle is the complete opposite. We're all on iMessage (except for one group of extended family on Messenger), and we like it that way. I was the last holdout for years while I went from Android -> Windows Phone -> Android -> iPhone.
You've listed a whole bunch of alternatives available to you, but for some reason you demand that Apple change its unique offering into just another one of those for you. Why? Is that not a completely enforced monoculture?
Apple has always been off to the side, doing their own thing, and for some reason that fact utterly enrages people. They demand that Apple become just like everyone else. But we already have everyone else! And in every single field Apple is in, there is more of everyone else than there is of Apple.
Have you considered people like Apple products precisely because they're not like everything else? That making Apple indistinguishable from Facebook or Google is no victory, but a significant loss for customer choice?
I have been an Android user for the past 15 years, and somehow iMessage has never been a problem. Most of the time I don't even know if someone uses iMessage or not.
Thanks to Apple co-opting phone numbers, there's literally no need to ever have iMessage for anyone
The machine I'm using now represents my choices and matches what matters to me, and works closer to perfectly than all my machines in the past
And yes, I have worked with macs, and no, the UX and the entire tyranny in the Apple ecosystem was not something I could live with
And yes, this machine is fast, predictable, a joy to work with and is a tool I control, not a tool to control me. If something happens to it, I can order the part with the same price that goes into a new machine, and keep using my laptop
Like, for phones, I want a phone which runs Linux, has NFC support, and also has iMessage so my friend who only communicates with blue-bubbles and will never message a green-bubble will still talk to me. I also want it to have regulatory approval in the country I live in so I can legally use it to make calls.
Because apple has closed the iMessage ecosystem such that a linux phone can't use it, such a device is impossible. I cannot vote for it.
As such, I will complain about every phone I own for the foreseeable future.
I actually agree with you, but I also suggest getting better friends.
And then they have iMessage, aka blue bubbles, which are kinda like Signal or Whatsapp or Telegram. Everyone in Europe uses whatsapp, and a lot of people in the US use iMessage. If you don't use whatsapp in europe, you'll have a rough time communicating with some social groups, and the same thing for iMessage in the US.
However, unlike every other messenger app I can think of, iMessage isn't cross platform.
Also unlike every other messenger I can think of, it comes installed by default and for some reason uses the same app as the SMS app, and also claims encryption but randomly switches to SMS and breaks encryption making it obviously the least secure of all the apps (and also backs up your keys to iCloud in a way apple can access them by default, neither here nor there).
Blue bubbles are when iMessage is acting as the iMessage app, and has encryption and can use features like sending high resolution photos, location, invites, and a bunch of other apple-specific features.
Green bubbles are when the iMessage app has converted itself into the SMS and RCS app, and has a reduced feature set, like being unable to remove people from group chats.
It's frankly a quite confusing decision to have two quite different apps built into the same app and indicate which feature-set is active based on the color of a UI element. I think everyone would prefer if apple split it into the 'Messages' app (SMS + RCS) and an optional 'iMessage' app which doesn't come installed by default, but you can download on the app store from Apple. I'm frankly surprised the EU hasn't forced apple to show a prompt for "default messenger app" on startup with the options being "Whatsapp", "iMessage", etc etc, like they do for default browser.
No, I don't think anyone would prefer that. People on iOS like iMessage, not SMS + RCS. Nobody is confused by it, they all know that green bubbles means you're texting someone who doesn't have an iPhone. It works seamlessly, it's just annoying when you want to have along conversation with a friend on Android because it doesn't have any nice iMessage extras available – that's why people don't like green bubbles.
I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
Takes a standard PSU. However, Mac Minis don't have occulink. So you might be a bit limited by whatever USB C can do.
Now if Intel can get there Arc drivers in order we'll see some real budget fun.
https://www.newegg.com/intel-arc-pro-b70-32gb-graphics-card/...
32 GB of VRAM for 1000$. Plus a 500$ Mac Mini.
Article mentions: "Apple finally approved our driver for both AMD and NVIDIA"
Does not mention Intel (GPUs). Select AMD GPUs work on macOS, but...
Macs (both Intel and ARM) support TB, but eGPU only work on Intel Macs, and basically only with AMD.
Good news is for medium end gaming choices are solid, and CUDA works on AMD these days.
I own one of these, the cage is just a piece of plastic. Anyway, I don't think 80$ is that big of a difference here. I can't really afford a 4k Nvidia GPU. Intel is my only hope.
Brand is TH3P4G3. Egpu.io has decent eGPU comparisons.
I wouldn't want all that dust in my GPU fans, prefer that near my case fans. I also don't like it given I got cats and want to store/box hw. I do use the eGPU in the fuse box. If I had a larger house, I'd use a server rack.
I was recently in the market for an eGPU but for a different niche (not eGPU/eNPU/eTPU but getting a HBA via TB to connect a LTO-6 drive via SAS). I went for a Sonnet instead, very low profile and small. I also bought an Asus one. Slightly bigger, came with more fans but TB4 instead of TB3 on the Sonnet. The cages are aluminium. Those eGPU were second hand (also without warranty but quicker S&H than Chinese New Year) but came with PSU. As you also gotta buy a PSU for it which came with the eGPUs I mentioned. For me no biggie, as I got a decent PSU lying around.
One nice thing about the Sonnet eGPU boxes is that they use standard SFX PSUs that are inexpensive to replace if they fail.
For LTO, I'm cheap, and iSCSI over a dedicated 2.5 Gbps Ethernet link is fast enough for my aging FC LTO-5 drives and spinning rust backup disks.
Sleep broke across all OSs, if sleep didn't break the GPU wouldn't get powered on with the laptop. If one side lost power during an outage (the gpu side, the laptop has a battery..) it would require an elaborate voodoo ritual of cycling both of them on and off until they 'caught' each other. It would cause the rest of the USB ports on the laptop to reset and drop comms with peripherals once or twice a week, necessitating a rain-dance restart.
when Oculink first started showing up I gave up all together and just said "fuck it i'll try it again in a few years.".
It worked fine when it worked fine, but the patches in between were not worth my time.
I blame Dell and their thunderbolt controllers entirely for the issue, but it left such a bad taste in my mouth that I would have a really tough time buying the newest Sonnet box to try it out. Now I have a desktop machine and don't fall into that market.
I ended up throwing that card (an rtx 3xxx) into a dell rackmount and have been happy with that card ever since.
to your point though: the non proprietary PSU was a nice feature, but in reality the expansion card for PCI->Thunderbolt or whichever interface you're using can be bought on alibaba for like 20-30 bucks and the PSU is worth another 30-40 bucks , a generic white-label 650w. I think if I did it over i'd just do that and make an enclosure, but the Sonnet boxes aren't too bad a value by the numbers.
It would work just like a discrete GPU when doing CPU+GPU inference: you'd run a few shared layers on the discrete GPU and place the rest in unified memory. You'd want to minimize CPU/GPU transfers even more than usual, since a Thunderbolt connection only gives you equivalent throughput to PCIe 4.0 x4.
How big a bottleneck is Thunderbolt 5 compared to an SSD? Is the 120 Gbps mode only available when linked to a monitor?
That's why all the projects streaming models into the GPU from an SSD popped up recently.
for thunderbolt enclosures, consider going through the list - https://egpu.io/best-egpu-buyers-guide/#tb3-enclosures
zero idea about mac support so YMMV.
Or I could have totally misunderstood the role of Docker in this.
My read of everything is that they are using Docker for NVIDIA GPUs for the sake of "how do you compile code to target the GPU"; for AMD they're just compiling their own LLVM with the appropriate target on macOS.
I would definitely be into this if adding an egpu was first class supported.
Something like eNPU or eTPU seems more appropriate here.
I got an eGPU back in 2018 and could never get it to work. To the point that it soured me from doing it again.
These days for heavy duty work I just offload to the cloud. This all feels like NVidia trying to be relevant versus ARM.
Except it's done by a third group, tinygrad, so it's more non-nvidia people wanting to use nvidia hardware one Apple hardware, than "nvidia trying to be relevant".
For well over the previous decade Apple has not allowed newer nVidia GPUs (by not allowing drivers).
A seven year old GPU (e.g. VEGA64, RTX1080Ti) can still process more tokens/second than most Apple Silicon (particularly the lower-ends).
As discussed elsewhere, Apple MAX/Ultra processors are best-suited for huge models (but are not as fast as e.g. RTX5090).
>>Apple approves...
This is a big deal.
I hooked up a Radeon RX 9060 XT to my Feodra KDE laptop (Yoga Pro 7 14ASP9) using a Razer Core X Chroma (40Gbps), and the performance when using the eGPU was very similar to using the Radeon 880M built into the laptop's Ryzen 9 365 APU.
So at least with my setup, performance is not great at all.
On paper, TB4 is capable of pushing 5GB/s, which is somewhere between 4x and 8x of PCIe 3.0, while a 16x PCIe 4.0 link can do ~31.5GB/s.
For numbers about all PCIe generations and lane counts, see the "History and revisions" section here: https://en.wikipedia.org/wiki/PCI_Express
Edit to add: the performance I measured is in gaming workloads, not compute
First, you need to connect the display directly to the eGPU rather than to the laptop.
Second, you need to make sure you have enough VRAM to minimize texture streaming during gameplay.
Third, you'll typically see better performance in terms of higher settings/resolutions vs higher framerates at lower settings/resolutions.
Fourth, depending on your system, you may be bottlenecked by other peripherals sharing PCH lanes with the Thunderbolt connection.
Finally, depending on the Thunderbolt version, PCIe bandwidth can be significantly lower than the advertised bandwidth of the Thunderbolt link. For example, while Thunderbolt 3 advertises 40 Gbps, and typically connects via x4 PCIe 3.0 (~32 Gbps), for whatever reason it imposes a 22 Gbps cap on PCIe data over the Thunderbolt link.
Even taking all this into account, you'll still see a significant performance drop on a current-gen GPU when running over Thunderbolt, though I'd still expect a useful performance improvement over integrated graphics in most cases (though not necessarily worth the cost of the eGPU enclosure vs just buying a cheap used minitower PC on eBay and gaming on that instead of a laptop).
Using proprietary connectors.
> XHCI
Not on Lightning.
> AHCI
How exactly would Apple not support AHCI?
> Using proprietary connectors.
Not for the past decade; it's been no connectors for most products, but standard PCIe connectors for the Mac Pro, and NVMe over Thunderbolt works fine.
>> XHCI
> Not on Lightning.
Again, not relevant to any recent products. And I'm pretty sure you're misunderstanding what XHCI is if you think anything with a Lightning connector is relevant here (XHCI is not USB 3.0). You can connect a Thunderbolt dock that includes an XHCI USB host controller and it works out of the box with no further driver or software support. I assume you can do the same with a USB controller card in a Mac Pro.
>> AHCI
> How exactly would Apple not support AHCI?
This might be another case of you not understanding what you're talking about and are lost in an entirely different layer of the protocol stack. Not supporting AHCI would be easy, since they're no longer selling any products that use SATA, and PCIe SSDs that use AHCI instead of NVMe died out a decade ago. But as far as I know, a SATA controller card at the far end of a Thunderbolt link or in a Mac Pro PCIe slot should still work, if the SATA controller uses AHCI instead of something proprietary as is typical for SAS controllers.
For the same reason that Microsoft requires Windows driver signing?
Drivers run with root permissions.
Isn't that the whole point of the walled garden, that they approve things? How could they aim and realize a walled garden without making things like that have to pass through them?
Because third party drivers usually are utter dogshit. That's how Apple managed to get double the battery life time even in the Intel era over comparable Windows based offerings.
Modern Mac is Macintosh descendants and by contrast PC is IBM PC descendants (their real name is technically PC-clone but because IBM PC don’t exist anymore the clone part have been scrapped).
And with Apple silicon Mac the two is again very different, for example Mac don’t use NVMe, they use just nand (their controller part is integrated in the SoC) and they don’t use UEFI or BIOS, but a combination of Boot ROM, LLB and iBoot
Redirect: https://x.com/*
If Apple was in the high-end server market, I see no reason why the company I was working for would not be running macOS on Apple hardware as servers, instead of the fleet of Linux based servers they had.
Despite how easy Apple makes it, nobody is really using Macs as a server in production. Apple[0] is not using them as a server in production. They would need a radically different strategy to replace Linux, because their efforts on macOS still haven't replaced Windows.
[0] https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...