I've been running my macOS desktop in an ESXi VM for over a year now. I came from a hackintosh, but got tired of the fragility of some upgrades. Some notes on this setup:
- Apple supports macOS running inside of ESXi. Officially, only on Apple host hardware (i.e. a Mac Pro, which is on the ESXi compatibility list). Apple even ships a VMXNet3 network driver in macOS.
- For ESXi specifically, you have to patch the host to boot macOS on non-Apple hardware. There's an explicit "am I running on Apple hardware?" check in there. DrDonk's unlocker on GitHub is what you need.
- You can specify board-id, ROM/MLB, etc values via your .vmx file. iMessage and everything work for me, but I'm also using identifiers from a real Mac Mini I retired a few years ago. I expect this will still work okay with generated data if you follow hackintosh guides.
- I pass through a GPU and a USB controller card for input/output. For USB, find an FL1100 chipset card, like the Inateck models, which works out of box on macOS, Windows, etc. Recently I moved my boot drive to a passed-through NVMe for extra speed. My other drives are ESXi virtual drives backed by storage elsewhere on my network.
- Most NVIDIA GPUs will need the NVIDIA Web Drivers to work. Kepler generation cards are the last to work out of box, and the only to currently work out of box on Mojave. If you don't need hefty performance, a GeForce GT 730 is a cheap card. Apple and NVIDIA are still not getting along, so I expect this will only get worse.
- However, I've never been able to get an AMD card to initialize via passthrough. The PCI device is seen, but the EFI strings or whatever else Apple is using to determine initialization paths are wrong. I recently spent awhile trying to figure this out, going so far as to write my own SSDT tables for the device, but finally gave up.
- This particular guide is using Chameleon, which is considered an outdated bootloader. Clover is far more flexible, and will work in VM setups that don't natively implement SMC like ESXi. (For the longest time Clover would crash an ESXi VM, but this now works as of a month or two ago).
And finally, on the context of this whole thing--I switched from a real Mac Pro to a hackintosh years and years ago, when it became clear Apple didn't care about upgrading the hardware. They still don't. The last Mac Pro release was 2013 (the 2013 iPhone was a 5S!). They've been promising things will improve "soon". They're the world's most valuable company; if they actually cared, at the organizational DNA level, it would have happened by now.
I remember a long time ago (maybe close to a decade?) I was looking into this a bit because I was interested in iPhone dev, and the requirement that that toolchain use Mac hardware was hard for me to swallow. It still seemed fairly hacky at that point, and ultimately I wasn't willing to shell out hundreds of dollars for a dev license and hardware just to see if it was worth it and I wasn't willing to go that far into the weeds on a commercial closed source platform where any problem I might have would likely be ignored with prejudice.
It's good to know they support some form of virtualization now. Do you know it they support any of the consumer grade virtualization systems? Even if I had to pay for a full macOS license (I think it wasn't nearly as cheap then either) and a developer license, that would have probably been much easier to swallow, and could have led to more hardware investment from me later if things worked out.
ESXi has a free license, but it is a type 1 hypervisor (basically an OS that just runs VMs).
VMWare Workstation can be patched to run vanilla macOS installs too, but unless you're running it under a boot camp'ed Windows on real Apple hardware, it's a license violation: https://github.com/DrDonk/unlocker
You can't buy individual macOS licenses these days, as far as I know. Apple hasn't charged for macOS or updates for at least a few years now. Their license text includes references to volume licensing, but I guess that's for very large-scale companies with direct contacts.
Can you please elaborate on your GPU passthrough setup? I was under the impression that NVIDIA have explicit checks to see if you're trying to virtualize a consumer brand GPU and throw an error in that case.
I have a PC with a GTX 970 that I wanted to try a similar install on, unfortunately the i5-3570K in there doesn't support VT-d from what I can tell.
Those checks are at the driver level, and only for Windows (you can circumvent them by not exposing VM cpuid details to the guest).
On macOS I haven't done anything special. Kepler cards work out of box, and other cards just need NVIDIA's own web drivers installed. Even on real Mac Pro hardware, those drivers are notoriously bad. They're tied to explicit macOS updates, so most people use something like this to patch older versions if new releases have issues: https://github.com/Benjamin-Dobell/nvidia-update
From my experience with a GTX 950 and a GTX 1060, the macOS Web Driver from NVIDIA doesn't seem to care about virtualization, and it works fine (assuming the drivers and passthrough are configured correctly).
Windows is a different story, and I had to use a patched vBIOS[1] to get my 1060 working without Windows 10 throwing the dreaded "Code 43" error (when it detects virtualization and refuses to work). In my case, the CPUID workaround mentioned in another comment didn't help.
As for VT-d, I believe the unlocked "K" processors from Intel all have VT-d disabled for some reason.
OSX-KVM has been super helpful for recovering old sparseimage-backups I've created back when I was still using a mac. This is because sparsebundlefs-fuse Linux driver doesn't work with this file-format (yet) and I don't have any access to a Mac. (Especially since recovering takes at least a dozen hours).
I've been running a macOS VM on my Threadripper 1950X for awhile now. Passing through 8 cores with a GTX 1060 attached and it runs great. In the latest Ubuntu 18.04 LTS, no need to compile qemu from source either, the version from apt get works just fine.
Yeah, one has been able to do this for some time now.
A few years ago I had a setup where (and I admit much of this was for the aesthetic purity of it, not practical reasons), I used a legally purchased copy of macos, copied it unmodified to my VM server, and, with a decent amount of effort, and a patched qemu (for the SMC thingie -- i donno if by now the patch is in mainline), was enjoying a non pirated, non modified macos on a linux host.
I even got USB and BLE passthrough working, so I could use the mac VM for iOS app development.
It was quite effective.
A more challenging project I did later (which I probably deserve some derision for not documenting and sharing) is "How to develop, sign, and upload iOS apps entirely from debian linux, entirely with F/OSS, no VM, no Xcode, just good ol' emacs and cmake". IIRC, there were very little Big Contributions I did, but plenty of assembling a large variety of other projects, bug fixing them, and making an integrated environment.
You should document the ability to build, develop, upload and sign from Linux. Unless you think it will cause Apple to close the path to do so (I assume others have worked out something similar and it would be a shame to cut them off).
I had a client give me a meaty Mac laptop, and had no desire to carry two laptops. I imaged their install, and then formatted the machine, and ran their image in a VM, and treated the underlying machine as my own. Worked well for a little while
Cool! Glad it worked for you. (I'm mildly curious what the chipset was on the guest that MacOS was running in) But like, not curious enough that you should spend any effort to find out.
Because I think one of the toughest parts was getting QEMU to make a machine macos would be happy with.
> A more challenging project I did later (which I probably deserve some derision for not documenting and sharing) is "How to develop, sign, and upload iOS apps entirely from debian linux, entirely with F/OSS, no VM, no Xcode, just good ol' emacs and cmake".
Please write this blog post. I've no interest in running Mac OS for any reason other than to test software in it and use it as a developer account for cross-platform apps. This would be a huge boon to people like me.
That must have been a long time ago indeed because the last version you could 'buy' was removed from sale in 2013. But actually you didn't buy a copy, you bought a disc and a license to upgrade the software on one 'Apple branded machine'.
I don't care if you run unlicensed software but it's not 'legally purchased' or 'non pirated'. You're just fooling yourself.
Also, I think you might have not noticed what in my opinion was the most important detail.
I'm not aware of if it's still true for hackintoshes or OSX in KVM or whatnot, but previously, most of the solutions I saw involved steps which I felt, I can't find a better word for than... "Gross". Things like, "Oh, yeah, it's totally MacOS working in a VM.... as long as you modify these random parts of the operating system and replace these other files with a binary I made"
Having many, many a time pirated Windows in the distant past I have some (possibly completely irrational) aversion to using an operating system with Joe Hacker's random patch applied to it. I'll prefer to take upstream thank-you-very-much. :)
My choice to pay for a non-pirated version of MacOS had nothing to do with believing that somehow Apple deserves my money -- I did so because I knew if and once I could get it to work, it would be more... clean.
You have a partial point, and I may have been sloppy with my words. But on the whole, you do not, and while I might willingly fool myself about a great many things, this is not one of them. I'm well aware that the license agreement claims to forbid running it on non Apple hardware.
However, no -- the software WAS "legally purchased" AND "not pirated".
I purchased, from Apple.com, a mac-mini, then in the apple app store paid for a stand-alone install of macos. I believe it was $20.
"legally purchased" -- I paid for it through the legitimate channel (as opposed to, say, buying it from anyone other than Apple Inc. like ebay)
"not pirated" -- Piracy (in this, that is, not the captain hook kind) context means taking advantage of the fact that digital assets can be perfectly duplicated to escape paying for it. I paid for it. I didn't duplicate it.
As to the issue of the Apple's "license agreement". On this we may simply view the world differently. It is my viewpoint that once a product is sold the original owner no longer is entitled to a reasonable expectation of control over its use. I can buy a screwdriver labeled "Only for use with Lowe's(tm) screws", and use it on whatever brand screws I want, because it's my screwdriver, and the notion that Lowe's gets to exert control over how I use a tool after I've legally bought it is
I am aware that currently some interpretations of US law are not congruous with this. I frankly don't care. You have the right to believe whatever you want, but to me law does not define truth. Interpretations of law frequently codify things that are not.
Regardless of bills passed or the outcome of court cases corporations are not (yet!) sentient entities, in 1897 Indiana the ratio of the circumference of a circle to its diameter was not exactly 3.2 -- it was pi, just like today, and all of my family members are equally human beings as anyone else despite much older laws asserting that some of them were 3/5th.
"fooling yourself" is a waste of effort. I'm well aware -- I have no romantic delusions about, for instance my copy of IDA Pro or MATLAB for example. It's a tool I greatly benefit from, that I cannot even come close to affording, so I stole it. Sure, it's almost certainly the case that in the coming years I'll again be in a position to legally purchase several licenses for work use, but that doesn't change the fact that I still pirated them.
But no, I legally purchased my copy of MacOS, it is not pirated, and I do not believe in the waste of time that is fooling ones self.
Congratulations, you have fooled yourself. You don’t have to put a lot of effort into convincing me that you didn’t because it doesn’t matter, I don’t care.
Heh, it's just like before, you're almost there but....
No, no... While I'll admit the truth, that I certainly put more effort into attempting to be understandable than I normally do, I believe you missed the reason why. If you use simple theory of mind maybe then it'll make sense to you.
I've never tried to "convince you" as you say, what would I get out of it, and also, if someone like you is determined to see the sky as green, they'll see green no matter how many blue skies you show them.
I know i don't "fool myself", it's not actually even plausible enough to be taken seriously, like, i donno, like if i suddenly claimed to be the best pitcher in the NL. :)
but everyone knows that, and I bet you do too.
What I was hoping I could help you see is there is a difference between a pedantic strict ruleset, like, say, laws, and human truths. and that we are to be their masters, not the other way around. Because if you go through life only seeing things through the lens you want to see, you'll miss out of a lot of cool stuff, not to mention, if past history means anything, often ending up on the wrong side of it :/
I donno, I do admit that I have the aspie trait that it's hard for me to just smile and nod to make someone go away, because i think (well, most) humans deserve respect. What you're misinterpreting as "effort to convince" I think the correct term would be is "altruistically try to help". I don't know you, but that doesn't mean I wouldn't feel bad at least trying to help.
Anyway, best, regardless.
maybe try to remember -- fooling yourself is being in denial about something, I think if you reread what I said before it'll click that that's not going on.
I really want this. I'm a hardcore linux guy(23+ years), but I have to run a windows vm for some instruments I have that wont run under wine. I hate windows. It sucks so much. Even just using it not that often with my instruments reminds me why I stopped using it.
I'm very similar. other than you've got about 5 more years on me :D
If you're referring to musical instruments, then yeah, that's the only reason I even have a windows 7 VM around. Although increasingly I've been able to get the necessary code to run under wine....
But yes:
> I hate windows. It sucks so much. Even just using it not that often with my instruments reminds me why I stopped using it.
Is basically my mental state as well. (and, hence, swore off windows around 2000)
I'm curious though, how would having a pure (aka linux) dev environment for macos help you in your situation?
I really want this. I'm a hardcore windows guy(23+ years), but I have to run a linux vm for some programs I have that wont run under WSL. I hate linux. It sucks so much. Even just using it not that often with my programs reminds me why I stopped using it.
My memory from having gotten an OSX guest running in Virtualbox on an OSX host was that video performance was pretty terrible. Virtualbox seems to only let Windows guests have real access to the GPU? The KVM/QEMU approach here sounds more promising along those lines (though maybe not on OSX hosts).
How does it compare to running a Hackintosh? Is it much easier to set up? Do you still need to have compatible hardware and to mess with drivers and system configuration? How easy are OS upgrades? How is the performance, including graphics? Any issues?
I've set up several Hackintosh machines in the past (bare metal / no hypervisor), but lately I've been using QEMU/KVM almost exclusively - specifically through Proxmox (I tried ESXi for a while, but I prefer the flexibility of QEMU/KVM). Most of the details from the linked guide can be adapted for Proxmox fairly easily.
For me, the killer advantage of using a hypervisor is the ability to easily create and restore snapshots. This eliminates the worry of macOS updates making your system unbootable, and allows you to test Kexts and other configuration changes (with any Hackintosh system, you're likely to need some trial and error to get things working after major updates) without fully committing and possibly getting stuck with changes that are difficult to reverse.
It only takes a few (dozen) times booting into single-user mode and trying to revert configuration changes to realize that snapshots are incredibly awesome :)
Hardware compatibility is still important, though it mostly boils down to needing a CPU with VT-x or AMD-V enabled, along with a compatible GPU for pass-through. I've had good luck with NVIDIA GeForce GTX and some older AMD Radeon cards (AMD cards needed a DSDT tweak). Audio over HDMI can be a little tricky, but I've gotten it working with every GPU I've tried. Those challenges are mostly the same whether you use a hypervisor or not.
Performance-wise, it's been pretty comparable to native performance, at least for general every-day use as a home computer for the family (streaming video services, casual gaming, some web/app development, etc).
> How does it compare to running a Hackintosh? Is it much easier to set up?
I have no idea about Hackintosh, but you can efficiently just grab ready-to-use VM image somewhere and use it with libvirt/virt-manager, e.g convert VMWare one. It's just works.
> Do you still need to have compatible hardware and to mess with drivers and system configuration?
I only had issues with sound since QEMU doesn't have compatible sound hardware and HDA driver for Hackintosh didn't work out-of-the-box. This can be solved by passing USB audio card.
Addition: obviously your CPU must support whatever instruction set macOS require otherwise it's won't work.
> How easy are OS upgrades?
There were some breakages, but they usually fixed within QEMU upstream in few weeks. Though this mean you might need recent QEMU sometimes.
> How is the performance, including graphics?
Performance is great. Surprisingly without GPU you can even run some image editing software with lag since Apple software OpenGL implementation is more or less working.
Or you can just buy compatible GPU and pass it to VM, then it's also just work without any issues.
> Any issues?
My VM image HWID got banned from using App Store fairly soon. Fortunately Xcode and all software I possibly need can be downloaded without it so I never did any research on how Hackintosh users solve it.
> My VM image HWID got banned from using App Store fairly soon
I never heard about HWID on Mac OS X before, and quick googling didn't produce any information about HWID on Hackintosh. Are you sure that is the real issue? Where can I read more about HWID?
Sorry. I probably shouldn't have called it HWID, but the fact is: App Store account that I've registered using said VM was after about a months limited so I unable to use it.
So I came to conclusion they detected it's was used on VM and limited it.
PS: And yeah I tried to use another new store account that also stuck in same state.
I guess it's because you used the same image as many other people?.. Which brings me to the question: was it an install disk image, or a pre-installed, ready-to-go OS image? Can you install a fresh OS from an install disk?
Someone posted a few months ago a story where they were using something similar to this (couldn't find the actual comment, sorry) where they had a HD with a very bare linux install on it that had enough drivers on it to boot on most any modern machine, load up qemu and boot a OSX image.
It was basically a "portable" hackintosh that could be moved between most any desktop machine.
I thought it was a cool idea, and tried to get it working with this repo, but didn't have much luck even with a full load distro (Ubuntu) on a Dell i5 with a nVidia GPU I had sitting around.
I had various success with this repository, unfortunately the various XMLs for libvirt and the "sh" boot scripts all reference different paths and capitalizations of files (e.g. even for the firmware committed to the repository etc).
However I did get it to work successfully with High Sierra at one point.
- use usb-tablet (<input type='tablet' bus='usb'/>) for much more convenient mouse input that does not lock to window. Initial setup may need to be done with usb mouse (<input type='mouse' bus='usb'/>)
All of the above need to be reflected in QEMU command line.
I've been using this setup for last half year without issues (mostly heavy compiling).
I'm looking forward porting this setup to 32-core Threadripper. Would be a hell of a beast that outperforms Apple HW that costs several times more.
You'll have much better chances if you can just buy external GPU since there are laptops with working IOMMU and there are people who got external GPU working in VM. Then for macOS it's going to be just normal PCI(e) device.
Unfortunately (almost?) all mobile GPUs won't even initialize within VM. I'm not expert in hardware side of things, but I suppose laptop drivers expect specific topology since mobile GPU almost never have own connection to display and sometimes don't even have own ROM. Even if somebody will find a way to make passed mobile GPU work on Windows VM it's very unlikely same trick will just work for macOS.
Theoretically you could run Linux headless on a Dell Latitude laptop, configure it to use the Dock's serial output for the main console, and then have it automatically boot the OS X VM with GPU pass through to the main screen. When you close the lid, Linux goes to sleep instead of OS X. You could disconnect it from the dock and just carry it around with you like a normal OS X machine. Open it up and OS X is there waiting for you. Connect it back to the dock and debug with another computer over serial and/or SSH whenever you need to troubleshoot.
> and then have it automatically boot the OS X VM with GPU pass through to the main screen
What GPU would that be? If it's integrated Intel there is very good chance it's wouldn't work. Last time I tried to get my Intel HD working in macOS VM attempt failed completely. As guys from Intel explained somewhere on mail list their Mac driver stack is quite different from what is there on Linux and Windows. This is why they for instance don't support their graphics virtualization (GVT-g) for macOS.
And for mobile GPU like Nvidia / AMD there own set of problems why they don't work at all in VM.
(iii) to install, use and run up to two (2) additional copies or instances of the Apple Software within virtual operating system environments on each Mac Computer you own or control that is already running the Apple Software, for purposes of: (a) software development; (b) testing during software development; (c) using macOS Server; or (d) personal, non-commercial use.
The grant set forth in Section 2B(iii) above does not permit you to use the virtualized copies or instances of the Apple Software in connection with service bureau, time-sharing, terminal sharing or other similar types of services.
Legal issues, I bet. AFAIK there's no way to run macos without Apple hardware and comply with the license. You could run a VPS provider on Mac Pros I guess, but I doubt it would be cheap.
I really doubt synthetic benchmarks are actually represent much since really I just tested it and some things are just weird. For instance memory bandwidth is only significantly lower in multi-core test which could be caused might be by both specific of Penryn CPU or some quirk in how Geekbench implement the test.
Additionally I just want to point out this is test done on absolutely default Ubuntu 18.04 with no VM optimizations whatsoever. It's totally possible to significantly improve VMs performance:
* By installing / compiling different kernel: low latency preemption option, 1000 HZ timer frequency, etc.
* Use HugePages to decrease memory fragmentation.
* Play around QEMU own options since right CPU options and host / guest balance with CPU pinning are helpful.
I used to improve my CPU-bound games performance on Windows VM a lot by doing some some fine tuning and I suppose macOS not much different.
On systems with a good support, it works great - it's transparent to the guest. I've been using it for around a couple of years, both with videogames and Lightroom.
You can't, unless you run qemu there. Which you can do, but it kinda defeats the purpose. And in the best case scenario you still gonna need some changes to host, like extra kernel boot parameters most likely.
It might be interesting if you can get the linux->darwin kernel and system library translation layer good enough that the binaries in the docker image are the macos programs, NOT qemu. But that's still far off, I think.
Though maybe you can enlighten me -- why the heck would you want to? It's in a VM -- you already have fantastic isolation. Why clutter it with an extra layer of packaging that to my naive eyes adds no more value?
Like, a cool hack might be running macos...._slowly_.... on a SBC like a raspi :D But when you have real virtualization, what's the use of docker? (honest question)
Docker uses a single Linux kernel for all containers. Docker containers are much lighter weight than VMs, since each VM must run a full kernel instance.
Yes, that's what I said. (although in a comment below this one. ;))
A container is much lighter weight than a VM, yes. (and so is a chroot)
But the MacOS userspace isn't running on the linux kernel, it's running on darwin. So, macos-in-docker would be an entire qemu installation in docker, with a macos image inside that.
EDIT:
Ahhh, because it doesn't make sense to run a full operating system in a container, only a single program, I'm guessing that what OP really wants is some way to dockerize a specific (or several) macos apps. To do so, you'd need a way to wrap the linux kernel do it can provide a darwin like api, as well as the same for any required libraries.
They made a MacOS to Linux translation (not virtualization) layer of sufficient quality that you an run a decent amount of (text only) native binaries.
Docker offers a simple way to provision the VM - install software, configure networking, execute apps... Docker itself is not a VM or isolation, it uses other tools for that (linux containers in the past, not sure how it works today), it's value is in the ease of use provided by its tooling.
I'm familiar enough with Docker to know of it as a combination of lxc, cgroups, and probably other things so that I can have 1 machine, 1 kernel, and yet multiple userspaces. These userspaces are not (as i understand it) Securely Isolated from eachother, but enough so that if there existed some monstrosity of a complex piece of software, which required lots of dependencies and customization, it might make sense to put it in a chroot, or a docker for the CoW benefits.
But what I'm not following (and again, I don't get the point of Docker, I don't use it, so in trying to learn I'm assuming you must know more...) is how it assists provisioning the VM as you say. Sure, it could _change_ the provisioning of the _host_ (i'm calling the inside of the docker container the host in this context). But it's not like the binaries being executed in the container is the mac operating system. It's a VM that within THAT is the mac operating system.
If I have mac running on a VM on a linux host, I still need to log in to that mac guest to configure networking, execute apps, install software.... So how did adding docker to the picture make it easier?
You're able to create files in Dockerfile, that could be used for configuration + the stuff about networking. AFAIK macOS has a textmode as well that should work similarly to other Unixes, but I'm not sure about that, but if so, you should be able to execute commands just like with a Linux VM. Yes, there'd need to be a bridge between the macOS VM and the Linux docker container.
If you read TFA, you'll see you can get the .app file from the App Store and then turn it into an ISO with a script that's included. But you can also use your favourite torrent site, like The Pirate Bay, to get an ISO.
Le sigh, continuous integration of osx and iOS apps is near impossible using standard cloud providers. I would gladly take an osx geared towards the data center.
I think you may have to pay them to use it for closed source stuff. Not sure about setting it up to run on-site or whatever.
Setup was sort-of quite easy, because I just ignored their CI system, however it works, and made the build run the same Python script I already used on my own Mac to make .dmgs - but it was still a bit painful in places, as their deployment options had some limitations, and the documentation wasn't always clear. But the end result does what I'd hoped for: master gets built, packaged, and uploaded to web site; build branch gets built, packaged, and pushed as a GitHub release.
(For Windows I use AppVeyor. Mostly similar experience, but one advantage: they'll host your build artefacts for you, though presumably up to some limit. Good for the non-release day-to-day builds, as I like to have a decent set of these but don't really care about keeping them indefinitely.)
We have a pretty good CI setup for our iOS apps using (shockingly) Microsoft's TFS. Not sure about any other cloud providers, but I was surprised by how easy it was to get working on a MS product.
If you find me funding, I could give you CI for mobile app development, a decent MVP, in 3 months (which is me taking how long i think it takes and multiplying by 3) :P
But, like a previous poster said, it _WOULD_ be in violation of Apple's ToS. So a legal budget would be required.
We actually need something like this internally for non mobile app reasons. We build telepresence.io at Datawire and macOS dev and testing is a huge pain. Interested in a job :) ?
Heck, if it's just iOS development, it might be possible to do a cloud CI that doesn't violate the ToS. When I was building all my iOS apps from Linux, I wasn't using osx.... I was using cross compiled clang....
- Apple supports macOS running inside of ESXi. Officially, only on Apple host hardware (i.e. a Mac Pro, which is on the ESXi compatibility list). Apple even ships a VMXNet3 network driver in macOS.
- For ESXi specifically, you have to patch the host to boot macOS on non-Apple hardware. There's an explicit "am I running on Apple hardware?" check in there. DrDonk's unlocker on GitHub is what you need.
- You can specify board-id, ROM/MLB, etc values via your .vmx file. iMessage and everything work for me, but I'm also using identifiers from a real Mac Mini I retired a few years ago. I expect this will still work okay with generated data if you follow hackintosh guides.
- I pass through a GPU and a USB controller card for input/output. For USB, find an FL1100 chipset card, like the Inateck models, which works out of box on macOS, Windows, etc. Recently I moved my boot drive to a passed-through NVMe for extra speed. My other drives are ESXi virtual drives backed by storage elsewhere on my network.
- Most NVIDIA GPUs will need the NVIDIA Web Drivers to work. Kepler generation cards are the last to work out of box, and the only to currently work out of box on Mojave. If you don't need hefty performance, a GeForce GT 730 is a cheap card. Apple and NVIDIA are still not getting along, so I expect this will only get worse.
- However, I've never been able to get an AMD card to initialize via passthrough. The PCI device is seen, but the EFI strings or whatever else Apple is using to determine initialization paths are wrong. I recently spent awhile trying to figure this out, going so far as to write my own SSDT tables for the device, but finally gave up.
- This particular guide is using Chameleon, which is considered an outdated bootloader. Clover is far more flexible, and will work in VM setups that don't natively implement SMC like ESXi. (For the longest time Clover would crash an ESXi VM, but this now works as of a month or two ago).
And finally, on the context of this whole thing--I switched from a real Mac Pro to a hackintosh years and years ago, when it became clear Apple didn't care about upgrading the hardware. They still don't. The last Mac Pro release was 2013 (the 2013 iPhone was a 5S!). They've been promising things will improve "soon". They're the world's most valuable company; if they actually cared, at the organizational DNA level, it would have happened by now.