How AWS Added Apple Mac Mini Nodes to EC2
servethehome.com
servethehome.com
1) After every stop/terminate of Mac instances, EC2 runs a scrubbing workflow on the underlying Dedicated Host to wipe the Mac mini's non-volatile storage and reset the NVRAM variables, to enable same security posture as any other EC2 instance. This workflow also upgrades the T2 chip on Mac mini to the latest BridgeOS version if needed. It may take 30-60 mins for this scrubbing workflow to complete, and up to 2-4 hours if BridgeOS update is required - during which the host shows up in "pending" state. We're actively working on lowering this scrubbing duration and really appreciate your feedback here. Important to note - You are not billed for any duration(s) during which the Mac1 Dedicated Host is in "pending" state (or any state other than "Available").
2. Once you have increased the size of the EBS volume on your Mac1 instance, you can execute following commands within macOS guest to increase the size of your APFS container.
1. Copy and paste the first three lines
PDISK=$(diskutil list physical external | head -n1 | cut -d" " -f1)
APFSCONT=$(diskutil list physical external | grep "Apple_APFS" | tr -s " " | cut -d" " -f8)
sudo diskutil repairDisk $PDISK
2. Accept the prompt with "y", then paste this command
sudo diskutil apfs resizeContainer $APFSCONT 0
Since the EBS volume was resized after boot, an instance reboot is required before the additional disk size is available for your use.
Every single mid-sized IT shop in the world has an urgent and valid use-case to virtualize OSX in an efficient and portable manner. Most individual power-end-users have similar use-cases.
How many hours / dollars / gigatons-of-carbon / calories are wasted on this comically inefficient, user-hostile and gratuitously complex state of affairs ?
Many, many more that are being "saved" by them not bundling chargers in iPhone. Kind of shows you where they stand.
Follow the $$$.
It would have been more likely for AWS to give specs to Apple, who could have then created a custom job for some extra monies.
In reality though, using Mac Minis is perfectly reasonable. It's likely the cheapest option for both Apple and Amazon.
Apple just doesn’t want to support it, and for developers, they want them to buy Macs to work on and only use VMs when absolutely necessary.
Apple sells integrated hardware and software. They don’t license macOS to anyone else. Why would they start now having seen that strategy fail in their corporate history already?
Jony Ive rolling in his grave.
I’m curious. Did nobody at any point in time ever say ‘this is unacceptable’ about this? Because it sure sounds so to me. It’d be (much) faster to boot up a new instance than wait for your previous instance to exit pending state.
This is a genuine question, I'm not being snobby or anything like that.
https://www.zdnet.com/article/why-apple-had-to-kill-the-xser...
Apple toyed with becoming more enterprisey at various times but their ultimate success came mostly from BYOD and enterprises proving Mac options because that's what many of their employees wanted. They do have business sales but it's still more of a pull than a push thing.
AWS was nothing if not a "B2B play" from the beginning. Having developed their own expertise with data centers and distributed computing, Amazon proceeded to sell that to other businesses.
In fact, as far as AWS is concerned, Amazon is just another big customer — and not even the biggest.
Now, Xserve didn’t work for a lot of reasons, but Apple exited the server market, and even stopped making the Server variant of macOS (even though it was just a handful of utilities at the end), largely because I think it realized two things:
1. It could never truly compete with Linux (and to a lesser degree, IIS) without making significant tradeoffs that are anathema to Apple (lowering prices significantly, giving up usage control, and decoupling vertically integrated hardware/software).
2. The cost of truly making a B2B play would require resource investments and commitments into an area Apple doesn’t need to care about and that would come at the expense of areas that are both more profitable and less of a PITA.
Apple is the richest company in the world and amongst the most profitable tech companies, why would they bother with enterprise when they can allow partners to deal with the integration side and focus on doing their stuff their way.
(There is one notable instance where Apple embraced enterprise, and that was when Apple added Exchange support in iPhone OS and admittedly, this was a huge deal and was ultimately a fatal blow to Blackberry. But it’s the exception that almost proves the rule. Apple added support for Exchange but left it up to others to figure out the broader MDM situations.)
Having said all that, I have always been surprised Apple hasn’t offered some sort of build/test service themselves and integrated it into TestFlight. It could be a driver for the all-important services revenue (in my imagination, the annual developer fee would cover a certain number of build minutes a month and then additional minutes would cost money). I have to think that such a solution is either in the works or that Apple looked at the support challenges and just decided to let other people do it at scale for them.
The thing is that much of the cloud is very much a commodity and Apple has successfully made its products and services anything but a commodity.
Yep. There have been a couple of specific things. Exchange and now MDM profiles. Probably a few others. But minor tweaks to remove major enterprise blockers. Small investments to remove complete showstoppers is just common sense.
>Apple hasn’t offered some sort of build/test service themselves and integrated it into TestFlight
Seems pretty logical. Who knows? It's certainly the trend with container platforms generally.
I may not be the typical end user, but I’m also not a business.
I predict they will build customer-facing 'cloud-powered features' where 'third party app code' can run on the apple cloud infrastructure 'on behalf' of a particular apple customer.
They will present a utility computing model to customers -- where each customer sees 'cloud-computing usage charges' displayed by app.
Ultimately, the cloud-compute/cloud-storage charges will tie back to the customer associated usage costs for storage or some other abstract usage based model of for cloud resource costs.
"Apple Cloud: build features for apple customers using usage-metered cloud resources -- with the customer paying for the usage charges they incur."
Unfortunately, Apple makes it somewhat uneconomical to use Macs in small bursts, as their license requires that Macs are leased for at least 24 hours.
The reason Apple doesn't do well B2B is nothing about their technology or marketing, it is likely all about their business operations. Apple is notorious for driving their own product-development lifecycle with heavy priority on their own design language. This is completely the opposite of how B2B sales work, where it is not uncommon for some development cycles to be dictated directly by a few (or even one) customer. I can't see Apple playing well with these expectations.
Development and sales (and support). Look at how many SKUs competing laptop manufacturers have because enterprises will specify some combination of ports and memory and CPU, and if the manufacturer doesn't have that particular combination, there will be no sale.
(Having said that: it's not obvious to me whether this is still as much of an issue as it used to be.)
- They would be vertically integrated with Apple Silicon
- Apple themselves are increasingly depending on cloud services
- Async/Await in Swift will likely land next year, making Server-Side Swift much more appealing
- Apple is greatly increasing cloud / Kubernetes hires
- Could share a single Apple Silicon ARM architecture from client(ios) to dev(mac) to cloudApple, as a company, does not "do" long-term support; they're the anti-Microsoft when it comes to backwards compatibility. As soon as you start being an infrastructure service provider that means you HAVE to offer an LTSB branch of your OS, and Apple does not want to do that (and macOS has stagnated a lot since Apple devoted most of their OS engineering efforts to iOS). Apple doesn't offer a "real" server OS SKU at all (the "macOS Server" app package is for SOHO management, not for use as a high-availability application server OS).
...which means that Apple would have to provide an offering using Linux or BSD - such as a "naked" Darwin distro - but again, Apple does not want to have to support that, and I'm sure devs don't want to see the IaaS OS scene fragment further, and Darwin is far, far removed from being yet another Linux distro.
Amazon offers ARM cloud systems - why should Apple risk their profitability by competing in an arena that doesn't pose any risk to their bottom-line? And what do they have to gain?
I would love to see Apple have an LTS branch of macOS, but I just don't see this happening. Too much would need to change in the company for that to happen.
From a macOS developer's standpoint, it's really a huge PITA there is no LTS. But, we adjust. Because we have a low-level product (compiler), we sometimes get into a situation where we need to sunset a product before we would normally do that, just because the older product doesn't run on the supported macOS versions. Doesn't happen often, but it does happen.
Obviously would work as FaaS or PaaS.
I think they don't want to cannibalize Mac sales. If you want to make an iPhone app, you have to buy a Mac to do it. They don't want to make it easy for you to rent one just to compile the app.
> A "lease period must be for a minimum period of twenty-four (24) consecutive hours."
I'm not sure if it's accurate, but I and many others see Apple as a fairly vindictive company, from stories about how Jobs operated to how they treat developers that speak out. My impression is that it's part of their DNA now.
[0]: https://news.ycombinator.com/item?id=25562532
So you won't be able to rent out one device to dozens of tenants per day who each use it for ten minutes or so while ensuring inter-tenant isolation.
Last, those Apple rules might as well be non-enforceable legally, but Apple might still retaliate by e.g. removing apps from their store that were uploaded by tenants of those EU companies.
Besides that, I forgot where it came out about the amount of money Amazon retail makes on it’s dedicated Apple store. Also, Apple and Amazon Video have special deals where you can do in app purchases of video within the app on iOS devices and the purchases are charged directly to your Prime account. No one else can do that for physical goods.
Apple and Amazon have been chummy since Jobs licensed the “one click patent” when the iTunes Music Store opened in 2003.
Yes, no, and yes.
Apple can't practically enforce that against you, an individual using a mac in your home. But then again you're probably not reselling your mac's functionality directly for profit to others. They for sure can seek to enforce it with AWS or others who sell it commercially. It's the reselling of their software and functionality for profit that is covered by the terms of the agreement.
> issue opened October 2013
> __ is broken on MacOS
> __ should fix it but I don't have a Mac to test.
> I do have a Mac but ___ can’t be replicated on my specific dev-tainted environment, plus I’m using macOS Y and you use macOS Z which is a pain for me to set up a VM for, if at all possible.
The raise of a few free macOS CI mitigated that a bit but the offer is honestly poor (except at CircleCI, props to them), and seems like Apple could care less so much they’re now actively making that harder than ever.
If Apple should be sued for anti-consumer behavior this is why... It costs time and money to fix software with every MacOS release and I'm getting close to suggesting we add a 50% Apple tax on products to pay for the additional support it entails.
As a matter of fact I went through the trouble to set up 10.9 through 10.14 in VirtualBox, driven by gitlab-runner via the virtualbox executor[0] (which is pure genius: it uses linked VM clones from a reference VM to spawn parallel runners, as well as using snapshots once SSH is ready after first boot for <2s spawn + full state reset).
Given the painful macOS on VirtualBox situation, at the time I wanted to move to VMware Fusion[1] but lacked time to implement an executor. It seems someone took that in their own hands and implemented a specific runner (in Swift!).
[0]: https://docs.gitlab.com/runner/executors/virtualbox.html
[1]: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1679
[2]: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1679#no...
Some companies only charge for commercial use of their products. So the “it’s free” argument doesn’t really hold water.
Companies have tried to “rent DVDs” digitally by playing the DVD on their hardware and streaming it. It got shot down by the courts.
I did find it interesting that Apple took the entire last run of that hardware for themselves, so that they could run their own internal VMware clusters on real Apple hardware for their developers.
There’s no way that Apple could build an AMI to run on hardware designed and built by someone else. Apple has a small number of people who really know what they’re doing, and they get shifted around to whatever project Apple thinks is most important — whether that’s hardware or software.
Apple needs to own the whole enchilada here, for multiple reasons. So, they can sell Apple hardware to AWS for inclusion inside of AWS data centers. But it has to be real-deal Apple hardware, running real-deal Apple OS, even if it is under a hyper visor like VMWare.
Server OS seemed to be targeted at small businesses to run on a Mac Pro or Mac Mini connected to NAS and hosting internal applications. But it wasn't very different from regular OSX.
If I remember correctly they previously bundled icloud services where each user got a slice of storage on the cloud, so I could see them extending that model to a time slice of compute as well, hosted on some form of K8s. I think the previous model was too limited for most use cases, but it’s not fundamentally broken.
I don’t think the economics of the M1 with 32+ cores make sense unless they go down the route of either build a cloud platform and amortise the cost of the chip design over a larger number of devices (or a chip let approach, but I’m not sure this matches their general design approach).
Also I’m pretty sure they think they have a competitive advantage over traditional x86 and ARM vendors so they’ll likely want to strike while that still holds.
The average Apple consumer customer can make plenty of use of cloud storage. What use does the average user have for cloud compute resources?
Unless Apple starts offloading some processing to the cloud, and gives them some free, I'm not sure how regular users would use this, and offering it for free assuming the majority of people won't use it is just asking for some popular app to come along that takes advantage of that and screws up the economics. Not to mention people probably won't be happy to have a limit and pay for overage use for stuff that is likely provided by free on other platforms (if Google Photos prettifies your photos for free and Apple does it through metered compute with some given free, that's bad optics).
Ridiculous :-))
https://en.wikipedia.org/wiki/Xserve
With their new CPUs maybe it is time to bring them back?
I mean surely they are not going to license Dell to put a M1 chip in a rackspace computer.
I've read before on HN about people using Mac hardware in production, when they needed access to macOS graphics API, but since this only concerns a software implementation, and hardware was standard until M1 processors, it's reasonable to assume that Apple could port that part of OS to Linux, if they needed to use it from servers.
Why?
For one thing, the tcp stack is ancient, and easy to syn flood. (It was forked from FreeBSD in 2001ish, before FreeBSD added syncookies and syncache). Synflooding os x makes the whole network stack unresponsive, including other interfaces like localhost.
They're Apple after all, so they can build macOS to boot from a standard environment - UEFI, kernel that doesn't check the copy prevention string on SMBIOS, etc.
Us, mere mortals, are more restricted, as we don't have the keys of the kingdom.
My understanding was they ditched Azure.
Netflix doesn’t use AWS for video, Facebook doesn’t use AWS for most stuff, Google and Microsoft obviously don’t now because of competition but they didn’t before GCP and Azure were things.
I do wish they have less restriction on macOS usage timing. And a services for Web Developers to test their site on Safari. Which is currently impossible to do with buying a Mac.
I wonder how "usage" is defined here: for instance, if I run a managed service on a fleet of Macs in the cloud, which serves many end customers from each machine, is this allowed under the terms?
IANAL (and haven't read the full EULA) but probably. The 24 hour restriction seems to apply to leasing. I would assume that running a multi-tenant managed service (or, for that matter, serving web pages) from a Mac would not be seen as leasing.
Not necessarily. A Mac Pro would work well for single-tenant usage where you want to run a series of applications that can each burst to the whole machine but not take long to finish.
Are you serious?
Apple's single biggest event of the year is a developer conference. They aren't putting on week-long events for video editors.
They also provided a bunch of Mac minis to Mac Stadium for OSS developers to build on.
They added virtualization to the M1 (it isn't on the A series) and have HyperKit built into the OS is all specifically for developers.
Well, Apple isn't giving them away, but they do indeed sell them to Mac Stadium.
They could probably just kill that part and make enough money on iOS anyway.
That being said, I don't think Apple has enough people who would be interested in hosting on MacOS to justify an Apple cloud offering. I think they know that.
Is it really? Lots of devs use Macs, yes. But compared to the sheer number of regular, non-dev users, that has to be tiiiiny.
I found lots of stats on the internet about what percentage of developers use Macs, but none about how just many Mac users are developers, so it'd be fascinating to see that number of anyone can find it.
However, I don't quite believe that there really are 20m active developers on the Apple platform. Evans Data estimates that there is a total of 27m developers worldwide [3]. It's not plausible that 74% of them should be writing software for Apple platforms. On the other hand, not all developers using a Mac are developing for the App Store. A lot of them will be Web developers.
I wouldn't be surprised if developers' share of Mac sales is 30%. Maybe it's just 20%. I don't know, but it's definitely not tiny.
[1] https://techcrunch.com/2018/06/04/app-store-hits-20m-registe...
[2] https://www.macrumors.com/2020/01/13/apple-mac-shipments-dow...
[3] https://www.future-processing.com/blog/how-many-developers-a...
Developers fuel their platform. Without developers MacOS and iOS are nothing. Apple knows this.
That's the real target.
And you can see it by how they treat the development tools (stuff like this: https://news.ycombinator.com/item?id=25563276 or the many deprecations, for example: https://news.ycombinator.com/item?id=25562604). Or CI/CD setups using their tools.
People like iOS because of the software, both the OS, and the software which runs atop it. This is why WWDC is Apple's single biggest event of the year.
Likewise, people don't buy Macs because they are metal and have a shiny fruit on the backside. Developers were among the first to migrate to Apple back in the 00s when they discovered its Unix underpinnings made it great for doing web development.
This idea that Apple doesn't care about developers doesn't hold water. If they didn't care, they wouldn't have bothered to add Hypervisor support into the M series CPUs. This is a feature which has almost no purpose outside of software development or as a server platform and Apple doesn't make dedicated server hardware so where does that leave us?
As of this writing, only 1-2 of the comment threads are related to the actual subject of the article (why Apple doesn't produce a rackmount form-factor machine, why you can't virtualize macOS on commodity x86 hardware) and the rest are just about the fact of Macs on EC2 (why Apple doesn't run their own cloud, whether the pricing is cost-effective, etc.).
It appears aws is charging $1/hr with a 24 hr minimum which is similar to every other provider out there that is offering MacOS instances.
I was excited to see that AWS was offering these as I assumed it would be priced just like everything else - hourly with no minimums.
I literally need access a few hrs a month for submitting binary updates to the app store. And I suspect there are a lot of other users with this same use case which may explain the consistent policy of minimum monthly usage.
3. Leasing for Permitted Developer Services. A. Leasing. You may lease or sublease a validly licensed version of the Apple Software in its entirety to an individual or organization (each, a “Lessee”) provided that all of the following conditions are met: (i) the leased Apple Software must be used for the sole purpose of providing Permitted Developer Services and each Lessee must review and agree to be bound by the terms of this License; (ii) each lease period must be for a minimum period of twenty-four (24) consecutive hours;
However it seems that no version of macOS since Lion (2011) is available to be purchased standalone anymore.
It is a shame that Apple forces such a wasteful and inconvenient process but it is more profitable for them and apparently they are legally allowed to do so.
We test some cross-platform software with an installer, and there's automated tests for every Windows version since 7, dozens of various Linux distros/versions (including x86 and ARM), but we never got around to automating Mac largely because of the complexity with getting back to a "clean" environment (eg: what you get with a fresh EC2 image). Having relatively few users compared to all other platforms plus the cost being a couple of orders of magnitude higher, I doubt we'll take advantage of this. Too bad.
The 24 hours minimum is applicable to the allocation duration a Mac1 Dedicated Host, and not to the instances running on that host. Put differently - once allocated, a Mac1 Dedicated Host can only be released from your account after 24 hours. You can however can launch, stop, start, and terminate as many mac1.metal instances with fresh macOS AMIs on that host as you need while that host remains allocated to you.
Additionally, Savings Plan (https://aws.amazon.com/savingsplans/) on Mac1 instances can provide up to 44% savings over On-demand prices for longer term commitments.
How much per month? Around $720?
Given I'd expect a lot of these to be used as iOS build servers, it's worth noting that Azure DevOps gives you 1,800 build minutes on the free tier, which AFAIK you can use with their MacOS agents.
Just checked - yes it is after about 2 months.
Obviously the 24 hour minimum means you’d be better buying one for most CI uses, which I think is what most people want them for.
However I think I remember a image can that also manipulated used apple hardware for the metal api? They would still benefit from the ability to scale on at least a daily level.
https://twitter.com/WindexCowboy/status/1333876188385841158
> The use of the Dedicated Hosts concept keeps the license requirement decoupled from the instance life cycle. While you do need to allocate the Mac for a minimum of 24 hours, you can launch and terminate a fresh mac1.metal instance on that box as many times as you like.
See https://news.ycombinator.com/item?id=25562532 for a huge caveat: scrubbing the old instance takes anywhere from 30 minutes to 4 hours.
So there are time constraints that make this far from ideal.
I bought one on ebay for $350 2 years back, I've added 16 GB RAM and a samsung evo SSD. At a total cost ~$500, this old mac mini is a beast.
I had just assumed they were not user upgradeable. So knowing now that they are, I'll have to take a look again.
How legally enforceable clickwrap such as that is varies from country to country.
One thing I've been wanting to do for a while is integrate with iMessage. iMessage stores a local SQLite database of message history. A forever-running MacOS instance could sync that with The Cloud and open up some pretty cool applications.
I remember reading somewhere that renting a Mac mini node from AWS has the same rental policies as renting a physical Mac mini from a rental company.
It makes a lot of sense to have a 24hr minimum when renting a physical machine delivered to a workplace, but it seems weird to have the same policies when renting a Mac mini on the cloud.
> Apple requires companies lease hardware and software "in its entirety to an individual or organization," ensuring peak performance and a one customer to one machine setup. Lease periods must be 24 consecutive hours and customers need to review and accept licensing terms for all first- and third-party software.
https://appleinsider.com/articles/20/11/12/apple-outlines-de...
As other posters have said, it's a MacOS license limitation :-(
The purchase price of a machine is far from the only cost of adding a machine to an enterprise. Depending on how a company is set up, that 73 days could be as little as 10 or 20 days. There are a lot of other factors to consider, especially if (as it is where I work), only two departments uses Macs, and the rest of the company is Windows.
Seems like they could just make a special case-less sled with only minor tweaks to the existing mini design. Obviously they would need to build a sled/ server chassis too with networking and power delivery.
The mini absolutely is intended for use as a server. It's what Apple provided to Mac Stadium for OSS projects to build on. Very few Mac Pros are used for CI servers, they are mostly used for video and audio processing.
There's Orka too, but as far as I can tell, you have to run it in MacStadium's cloud https://www.macstadium.com/orka
https://www.youtube.com/watch?v=Pn3miC_tTH0&t=1m10s
"We have to maintain each individual device." There's an effete millennial joke in there somewhere.
Apple have made it so building/testing iOS apps requires access to OS X, and OS X can only be run on Apple hardware, so they've ensured there is some segment that requires access to Apple hardware.
It just seems that the market isn't there and people just go for full-machine MacStadium rental or use their own hardware.
[1] https://9to5mac.com/2020/11/11/macos-big-sur-adds-leasing-te...
Also, the "purposes" of Section 3 explicitly lists CI as "(B)", followed (confusingly) by "B.".
> For purposes of this Section 3: (A) End User Lessee means a Lessee who is the end user ultimately using the leased Apple Software solely for Permitted Developer Services; and (B) Permitted Developer Services means continuous integration services, including but not limited to software development, building software from source, automated testing during software development, and running necessary developer tools to support such activities. Each Lessor must provide Apple with advance notice prior to leasing or subleasing the Apple Software pursuant to this Section 3 by contacting Apple Developer Relations (https://developer.apple.com/contact/macos-license/).
> B. Subleasing. A Lessee may further sublease the Apple Software pursuant to this Section 3 provided that such Lessee complies with all of the terms of this Section 3. A Lessee subleasing the Apple Software (who shall also be considered a Lessor under this Section 3) must fully relinquish exclusive use and control of the Apple Software and the Apple-branded hardware on which it is installed to its Lessee during the lease period.
From which I understand that e.g 1. MacStadium is a "Lessor", 2. GitHub is a "Lessee" that does subleasing and has to lease for 24h minimum, and 3. a GitHub Actions user is a "End User Lessee" but could lease for fractions of 24h.
Strangely this means that GH cannot provide macOS on its own!
Also, this seems to mean that GH has to provide bare metal to its users, not VMs ("must fully relinquish exclusive use and control of the Apple Software and the Apple-branded hardware"), which is too bad given what you can do with VMs[0].
That's pretty cool; they bridge ethernet/IP (I assume) to SATA or NVMe on Thunderbolt.
If they actually wanted to make it easy to develop software they would allow compiling on other platforms, and ideally provide a VM to test on.
This is why I support IE and Edge on personal sites but not Safari. You can download free VMs from Microsoft for web testing, but you have to pay Apple to make things work on their platform.
My use case might be too specific. But, at home I have a Windows Desktop and Laptop. In the office I use a Mac Mini. Currently, I need to test Apple Pay for Safari. This requires test iCloud credentials. On a mac, this is difficult or prohibitive to swap between live iCloud and test iCloud logins. On Windows, testing via browser emulation is shaky at best.
Also, easily sharing access to a "cloud" (read: remote) instance of macOS with other developer staff is beneficial.
How quickly would Apple fold if all of a sudden there weren't third party applications for their devices?
Apple is very much biting the hand(s) that feed here.
It's 2020, this shit has been happening for a while, and it's no secret. I haven't figured out why a self-respecting developer would subject themselves to that nonsense.