Osx-kvm: Run macOS on QEMU/KVM
github.com
github.com
- Apple supports macOS running inside of ESXi. Officially, only on Apple host hardware (i.e. a Mac Pro, which is on the ESXi compatibility list). Apple even ships a VMXNet3 network driver in macOS.
- For ESXi specifically, you have to patch the host to boot macOS on non-Apple hardware. There's an explicit "am I running on Apple hardware?" check in there. DrDonk's unlocker on GitHub is what you need.
- You can specify board-id, ROM/MLB, etc values via your .vmx file. iMessage and everything work for me, but I'm also using identifiers from a real Mac Mini I retired a few years ago. I expect this will still work okay with generated data if you follow hackintosh guides.
- I pass through a GPU and a USB controller card for input/output. For USB, find an FL1100 chipset card, like the Inateck models, which works out of box on macOS, Windows, etc. Recently I moved my boot drive to a passed-through NVMe for extra speed. My other drives are ESXi virtual drives backed by storage elsewhere on my network.
- Most NVIDIA GPUs will need the NVIDIA Web Drivers to work. Kepler generation cards are the last to work out of box, and the only to currently work out of box on Mojave. If you don't need hefty performance, a GeForce GT 730 is a cheap card. Apple and NVIDIA are still not getting along, so I expect this will only get worse.
- However, I've never been able to get an AMD card to initialize via passthrough. The PCI device is seen, but the EFI strings or whatever else Apple is using to determine initialization paths are wrong. I recently spent awhile trying to figure this out, going so far as to write my own SSDT tables for the device, but finally gave up.
- This particular guide is using Chameleon, which is considered an outdated bootloader. Clover is far more flexible, and will work in VM setups that don't natively implement SMC like ESXi. (For the longest time Clover would crash an ESXi VM, but this now works as of a month or two ago).
And finally, on the context of this whole thing--I switched from a real Mac Pro to a hackintosh years and years ago, when it became clear Apple didn't care about upgrading the hardware. They still don't. The last Mac Pro release was 2013 (the 2013 iPhone was a 5S!). They've been promising things will improve "soon". They're the world's most valuable company; if they actually cared, at the organizational DNA level, it would have happened by now.
I have a PC with a GTX 970 that I wanted to try a similar install on, unfortunately the i5-3570K in there doesn't support VT-d from what I can tell.
On macOS I haven't done anything special. Kepler cards work out of box, and other cards just need NVIDIA's own web drivers installed. Even on real Mac Pro hardware, those drivers are notoriously bad. They're tied to explicit macOS updates, so most people use something like this to patch older versions if new releases have issues: https://github.com/Benjamin-Dobell/nvidia-update
Windows is a different story, and I had to use a patched vBIOS[1] to get my 1060 working without Windows 10 throwing the dreaded "Code 43" error (when it detects virtualization and refuses to work). In my case, the CPUID workaround mentioned in another comment didn't help.
As for VT-d, I believe the unlocked "K" processors from Intel all have VT-d disabled for some reason.
This was only the case up to Haswell CPUs. They stopped crippling IOMMU capabilities since Haswell Refresh (i7-4790K).
It's good to know they support some form of virtualization now. Do you know it they support any of the consumer grade virtualization systems? Even if I had to pay for a full macOS license (I think it wasn't nearly as cheap then either) and a developer license, that would have probably been much easier to swallow, and could have led to more hardware investment from me later if things worked out.
VMWare Workstation can be patched to run vanilla macOS installs too, but unless you're running it under a boot camp'ed Windows on real Apple hardware, it's a license violation: https://github.com/DrDonk/unlocker
You can't buy individual macOS licenses these days, as far as I know. Apple hasn't charged for macOS or updates for at least a few years now. Their license text includes references to volume licensing, but I guess that's for very large-scale companies with direct contacts.
* What's your ESXi host?
* How did you learn ESXi? Coming from VirtualBox/VMWare/Hyper V (Windows 10) the learning curve seems very, very steep.
For any one interested in using the Ryzen Threadripper for a Hackintosh VM, see https://www.reddit.com/r/hackintosh/comments/8knz91/threadri...
https://github.com/torarnv/sparsebundlefs/issues/7#issuecomm...
A few years ago I had a setup where (and I admit much of this was for the aesthetic purity of it, not practical reasons), I used a legally purchased copy of macos, copied it unmodified to my VM server, and, with a decent amount of effort, and a patched qemu (for the SMC thingie -- i donno if by now the patch is in mainline), was enjoying a non pirated, non modified macos on a linux host.
I even got USB and BLE passthrough working, so I could use the mac VM for iOS app development.
It was quite effective.
A more challenging project I did later (which I probably deserve some derision for not documenting and sharing) is "How to develop, sign, and upload iOS apps entirely from debian linux, entirely with F/OSS, no VM, no Xcode, just good ol' emacs and cmake". IIRC, there were very little Big Contributions I did, but plenty of assembling a large variety of other projects, bug fixing them, and making an integrated environment.
If you're referring to musical instruments, then yeah, that's the only reason I even have a windows 7 VM around. Although increasingly I've been able to get the necessary code to run under wine....
But yes: > I hate windows. It sucks so much. Even just using it not that often with my instruments reminds me why I stopped using it.
Is basically my mental state as well. (and, hence, swore off windows around 2000)
I'm curious though, how would having a pure (aka linux) dev environment for macos help you in your situation?
I don't know what it is, but I thought we've had that for some time now....
But how does a system that lets you compile iOS applications natively from linux decrease your windows usage?
Oh, or were you referring to the "clean room" osx-kvm-debian project, not the "ios build system on debian" project?
And if you'd tell me... maybe I'd help.... :P :)
I don't care if you run unlicensed software but it's not 'legally purchased' or 'non pirated'. You're just fooling yourself.
However, no -- the software WAS "legally purchased" AND "not pirated".
I purchased, from Apple.com, a mac-mini, then in the apple app store paid for a stand-alone install of macos. I believe it was $20.
"legally purchased" -- I paid for it through the legitimate channel (as opposed to, say, buying it from anyone other than Apple Inc. like ebay)
"not pirated" -- Piracy (in this, that is, not the captain hook kind) context means taking advantage of the fact that digital assets can be perfectly duplicated to escape paying for it. I paid for it. I didn't duplicate it.
As to the issue of the Apple's "license agreement". On this we may simply view the world differently. It is my viewpoint that once a product is sold the original owner no longer is entitled to a reasonable expectation of control over its use. I can buy a screwdriver labeled "Only for use with Lowe's(tm) screws", and use it on whatever brand screws I want, because it's my screwdriver, and the notion that Lowe's gets to exert control over how I use a tool after I've legally bought it is
I am aware that currently some interpretations of US law are not congruous with this. I frankly don't care. You have the right to believe whatever you want, but to me law does not define truth. Interpretations of law frequently codify things that are not.
Regardless of bills passed or the outcome of court cases corporations are not (yet!) sentient entities, in 1897 Indiana the ratio of the circumference of a circle to its diameter was not exactly 3.2 -- it was pi, just like today, and all of my family members are equally human beings as anyone else despite much older laws asserting that some of them were 3/5th.
"fooling yourself" is a waste of effort. I'm well aware -- I have no romantic delusions about, for instance my copy of IDA Pro or MATLAB for example. It's a tool I greatly benefit from, that I cannot even come close to affording, so I stole it. Sure, it's almost certainly the case that in the coming years I'll again be in a position to legally purchase several licenses for work use, but that doesn't change the fact that I still pirated them.
But no, I legally purchased my copy of MacOS, it is not pirated, and I do not believe in the waste of time that is fooling ones self.
No, no... While I'll admit the truth, that I certainly put more effort into attempting to be understandable than I normally do, I believe you missed the reason why. If you use simple theory of mind maybe then it'll make sense to you.
I've never tried to "convince you" as you say, what would I get out of it, and also, if someone like you is determined to see the sky as green, they'll see green no matter how many blue skies you show them.
I know i don't "fool myself", it's not actually even plausible enough to be taken seriously, like, i donno, like if i suddenly claimed to be the best pitcher in the NL. :) but everyone knows that, and I bet you do too.
What I was hoping I could help you see is there is a difference between a pedantic strict ruleset, like, say, laws, and human truths. and that we are to be their masters, not the other way around. Because if you go through life only seeing things through the lens you want to see, you'll miss out of a lot of cool stuff, not to mention, if past history means anything, often ending up on the wrong side of it :/
I donno, I do admit that I have the aspie trait that it's hard for me to just smile and nod to make someone go away, because i think (well, most) humans deserve respect. What you're misinterpreting as "effort to convince" I think the correct term would be is "altruistically try to help". I don't know you, but that doesn't mean I wouldn't feel bad at least trying to help.
Anyway, best, regardless.
maybe try to remember -- fooling yourself is being in denial about something, I think if you reread what I said before it'll click that that's not going on.
I'm not aware of if it's still true for hackintoshes or OSX in KVM or whatnot, but previously, most of the solutions I saw involved steps which I felt, I can't find a better word for than... "Gross". Things like, "Oh, yeah, it's totally MacOS working in a VM.... as long as you modify these random parts of the operating system and replace these other files with a binary I made"
Having many, many a time pirated Windows in the distant past I have some (possibly completely irrational) aversion to using an operating system with Joe Hacker's random patch applied to it. I'll prefer to take upstream thank-you-very-much. :)
My choice to pay for a non-pirated version of MacOS had nothing to do with believing that somehow Apple deserves my money -- I did so because I knew if and once I could get it to work, it would be more... clean.
Because I think one of the toughest parts was getting QEMU to make a machine macos would be happy with.
Please write this blog post. I've no interest in running Mac OS for any reason other than to test software in it and use it as a developer account for cross-platform apps. This would be a huge boon to people like me.
I have no idea about Hackintosh, but you can efficiently just grab ready-to-use VM image somewhere and use it with libvirt/virt-manager, e.g convert VMWare one. It's just works.
> Do you still need to have compatible hardware and to mess with drivers and system configuration?
I only had issues with sound since QEMU doesn't have compatible sound hardware and HDA driver for Hackintosh didn't work out-of-the-box. This can be solved by passing USB audio card.
Addition: obviously your CPU must support whatever instruction set macOS require otherwise it's won't work.
> How easy are OS upgrades?
There were some breakages, but they usually fixed within QEMU upstream in few weeks. Though this mean you might need recent QEMU sometimes.
> How is the performance, including graphics?
Performance is great. Surprisingly without GPU you can even run some image editing software with lag since Apple software OpenGL implementation is more or less working.
Or you can just buy compatible GPU and pass it to VM, then it's also just work without any issues.
> Any issues?
My VM image HWID got banned from using App Store fairly soon. Fortunately Xcode and all software I possibly need can be downloaded without it so I never did any research on how Hackintosh users solve it.
PS: Edited. Added note about SSE.
I never heard about HWID on Mac OS X before, and quick googling didn't produce any information about HWID on Hackintosh. Are you sure that is the real issue? Where can I read more about HWID?
So I came to conclusion they detected it's was used on VM and limited it.
PS: And yeah I tried to use another new store account that also stuck in same state.
> Can you install a fresh OS from an install disk?
Yes of course you can. It's explained on GitHub.
It was basically a "portable" hackintosh that could be moved between most any desktop machine.
I thought it was a cool idea, and tried to get it working with this repo, but didn't have much luck even with a full load distro (Ubuntu) on a Dell i5 with a nVidia GPU I had sitting around.
I haven't integrated GPU passthrough yet, though.
For me, the killer advantage of using a hypervisor is the ability to easily create and restore snapshots. This eliminates the worry of macOS updates making your system unbootable, and allows you to test Kexts and other configuration changes (with any Hackintosh system, you're likely to need some trial and error to get things working after major updates) without fully committing and possibly getting stuck with changes that are difficult to reverse.
It only takes a few (dozen) times booting into single-user mode and trying to revert configuration changes to realize that snapshots are incredibly awesome :)
Hardware compatibility is still important, though it mostly boils down to needing a CPU with VT-x or AMD-V enabled, along with a compatible GPU for pass-through. I've had good luck with NVIDIA GeForce GTX and some older AMD Radeon cards (AMD cards needed a DSDT tweak). Audio over HDMI can be a little tricky, but I've gotten it working with every GPU I've tried. Those challenges are mostly the same whether you use a hypervisor or not.
Performance-wise, it's been pretty comparable to native performance, at least for general every-day use as a home computer for the family (streaming video services, casual gaming, some web/app development, etc).
However I did get it to work successfully with High Sierra at one point.
Compatability?
- tell explicitly it's a "Penryl" CPU (<model fallback='allow'>Penryl</model>)
- force AES instructions in order to use encryption effectively (<feature policy='require' name='aes'/>)
- explicitly define topology (<topology sockets='1' cores='8' threads='2'/>)
- use usb-tablet (<input type='tablet' bus='usb'/>) for much more convenient mouse input that does not lock to window. Initial setup may need to be done with usb mouse (<input type='mouse' bus='usb'/>)
All of the above need to be reflected in QEMU command line.
I've been using this setup for last half year without issues (mostly heavy compiling).
I'm looking forward porting this setup to 32-core Threadripper. Would be a hell of a beast that outperforms Apple HW that costs several times more.
What GPU would that be? If it's integrated Intel there is very good chance it's wouldn't work. Last time I tried to get my Intel HD working in macOS VM attempt failed completely. As guys from Intel explained somewhere on mail list their Mac driver stack is quite different from what is there on Linux and Windows. This is why they for instance don't support their graphics virtualization (GVT-g) for macOS.
And for mobile GPU like Nvidia / AMD there own set of problems why they don't work at all in VM.
Unfortunately (almost?) all mobile GPUs won't even initialize within VM. I'm not expert in hardware side of things, but I suppose laptop drivers expect specific topology since mobile GPU almost never have own connection to display and sometimes don't even have own ROM. Even if somebody will find a way to make passed mobile GPU work on Windows VM it's very unlikely same trick will just work for macOS.
(iii) to install, use and run up to two (2) additional copies or instances of the Apple Software within virtual operating system environments on each Mac Computer you own or control that is already running the Apple Software, for purposes of: (a) software development; (b) testing during software development; (c) using macOS Server; or (d) personal, non-commercial use.
The grant set forth in Section 2B(iii) above does not permit you to use the virtualized copies or instances of the Apple Software in connection with service bureau, time-sharing, terminal sharing or other similar types of services.
I just Googled "mac os vps"
Not affiliated
Total score of my host against VM:
Host: 3954 single / 13713 multi core
macOS VM: 3770 single / 11630 multi core
* By installing / compiling different kernel: low latency preemption option, 1000 HZ timer frequency, etc.
* Use HugePages to decrease memory fragmentation.
* Play around QEMU own options since right CPU options and host / guest balance with CPU pinning are helpful.
I used to improve my CPU-bound games performance on Windows VM a lot by doing some some fine tuning and I suppose macOS not much different.
https://github.com/Cleafy/sxkdvm
Though maybe you can enlighten me -- why the heck would you want to? It's in a VM -- you already have fantastic isolation. Why clutter it with an extra layer of packaging that to my naive eyes adds no more value?
Like, a cool hack might be running macos...._slowly_.... on a SBC like a raspi :D But when you have real virtualization, what's the use of docker? (honest question)
But what I'm not following (and again, I don't get the point of Docker, I don't use it, so in trying to learn I'm assuming you must know more...) is how it assists provisioning the VM as you say. Sure, it could _change_ the provisioning of the _host_ (i'm calling the inside of the docker container the host in this context). But it's not like the binaries being executed in the container is the mac operating system. It's a VM that within THAT is the mac operating system.
If I have mac running on a VM on a linux host, I still need to log in to that mac guest to configure networking, execute apps, install software.... So how did adding docker to the picture make it easier?
Hence my confusion.
A container is much lighter weight than a VM, yes. (and so is a chroot)
But the MacOS userspace isn't running on the linux kernel, it's running on darwin. So, macos-in-docker would be an entire qemu installation in docker, with a macos image inside that.
EDIT: Ahhh, because it doesn't make sense to run a full operating system in a container, only a single program, I'm guessing that what OP really wants is some way to dockerize a specific (or several) macos apps. To do so, you'd need a way to wrap the linux kernel do it can provide a darwin like api, as well as the same for any required libraries.
OP: You may find this project interesting: https://www.darlinghq.org/project-status/
They made a MacOS to Linux translation (not virtualization) layer of sufficient quality that you an run a decent amount of (text only) native binaries.
It might be interesting if you can get the linux->darwin kernel and system library translation layer good enough that the binaries in the docker image are the macos programs, NOT qemu. But that's still far off, I think.
To my great surprise I found this on Amazon, and I'm curious about what others think is how apple hasn't demanded its removal: https://www.amazon.com/10-13-High-Sierra-Full-Install/dp/B07...
But, like a previous poster said, it _WOULD_ be in violation of Apple's ToS. So a legal budget would be required.
I think you may have to pay them to use it for closed source stuff. Not sure about setting it up to run on-site or whatever.
Setup was sort-of quite easy, because I just ignored their CI system, however it works, and made the build run the same Python script I already used on my own Mac to make .dmgs - but it was still a bit painful in places, as their deployment options had some limitations, and the documentation wasn't always clear. But the end result does what I'd hoped for: master gets built, packaged, and uploaded to web site; build branch gets built, packaged, and pushed as a GitHub release.
(For Windows I use AppVeyor. Mostly similar experience, but one advantage: they'll host your build artefacts for you, though presumably up to some limit. Good for the non-release day-to-day builds, as I like to have a decent set of these but don't really care about keeping them indefinitely.)