This is what I use to run QEMU on my M1 Air. I had a few bumps setting it up but now it runs flawlessly. I SSH to it via VS Code and basically I have a fully functional Linux dev environment with MacOS as the UI, just the way I like it.
Honestly, I haven't been this excited about a new setup since Moore's Law was in effect :-)
I do this except the Linux is on a separate box and the dev envs are docker containers.
Decided to spend my money on decent, upgradeable hardware instead of shelling out 5k for another MB Pro. My next MacBook will probably be the most low-end I can get.
VSCode's remote/docker support is a godsend. Seems like we're heading back to the dumb terminal paradigm.
Do you have your projects cloned in the remote host?
I ask because I was excited about this prospect only to learn that I could only use Remote: Containers if codebase and container are on the same machine as the project workspace is just volume-mounted in the container.
Yeah I dusted off an old licence I had for resilio sync and use that to sync with my local machine.
You can get around this by running git clone inside your container as part of the startup process. But 1. It can add quite a delay to starting a container depending on the size of your codebase, and 2. Not all of my code is necessarily committed and pushed.
For me, syncing works well but I can see how it would not suit every use case
Fair enough. I don't want to clone from within the container because I like the idea of being able to nuke it at any time and just start over from local code.
Probably similar reasons to what you mean by not having all code committed and pushed.
I have a Dell R720 that I bought fairly cheaply ~$600 came with 192GB of RAM. I shoved some SSDs in it with a GPU. Sits in my garage. If I'm not home I can still remote into my terminal via wireguard. Good deals can be found using lab gopher:
I saw some VPS providers offer hourly billing, which can get me a 64cores + 64gb ram at like $30 a month. I'm going to go with that. I actually don't have landline/fiber internet at home so your solution wouldn't work for me, but I'll keep it in mind I didn't know about such deals!
QEMU is amazing. I run Linux, ans being able to dockerize Windows with it, and set up a selenium testing environment I can run locally or on CI servers is genuinely mind-blowing. Headless chrome, Firefox plus a windows in docker setup means pretty great automated test coverage without relying on third party infra.
It (QEMU) actually uses the older https://developer.apple.com/documentation/hypervisor apis which has lower level control. QEMU provides device emulation and Hypervisor.framework provides access to CPU’s virtualization hardware. Virtualization.framework is the new macOS 11 api that emulates the entire system but has much worse device emulation support and also lacks framebuffer emulation (I think)
Also worth mentioning that UTM developers has been contribution patches to QEMU to get TCG (code emulation layer) to work under iOS / M1 Mac[1] as well (whereas I think TCG is still broken as of latest QEMU release due to mprotect[2]). So huge, huge thanks for this as well!
$ qemu-system-alpha -nographic
PCI: 00:00:0 class 0300 id 1013:00b8
PCI: region 0: 10000000
PCI: region 1: 12000000
PCI: 00:01:0 class 0200 id 8086:100e
PCI: region 0: 12020000
PCI: region 1: 0000c000
PCI: 00:02:0 class 0101 id 1095:0646
PCI: region 0: 0000c040
PCI: region 1: 0000c048
PCI: region 3: 0000c04c
>>>
This is a very nice frontend. Someone should definitely port it or rewrite it for Linux. All the existing UIs are either ugly or use libvirt, which is a nasty pile of hacks that has no place on a desktop machine.
Thanks but I think a port is virtually impossible as this was designed in SwiftUI and depends extensively on Apple APIs. GNOME Boxes is a good UI on Linux.
libvirt has put a lot of time into figuring out how to run QEMU in the most secure and performant way (specific low-privilege user, seccomp, all the right options for virtio, etc, etc). If you aren't using it then that's a lot of effort to reinvent, and you probably missed something out.
libvirt wasted a lot of effort into abstracting everything behind a ton of pointless hypervisor-agnostic cruft, only to then deprecate basically every single backend except QEMU.
I have used libvirt for almost a decade before going back to plain QEMU; I must say that using it from the CLI is an herculean task, you often have to edit plain XML in order to do whatever you want, and the cherry on top is that `virsh`is a huge pile of cow dung that often drags you into a Sisyphean fight against the aforementioned pointless abstractions - you find yourself configuring "domains" and "virtual networks" when you really only wanted to launch Windows to run Word.
When I had to run stuff on a big server which only does VM, maybe with OpenStack and other orchestration systems on top, libvirt was often _fine_. I still use it in those use cases (I usually prefer Proxmox this day, though), it works fine, I can connect to remote instances using SSH and Virt-Manager and I like how it often just works.
My use case, 90% of the time, is that I just want to run a damn Windows or FreeBSD VM on my workstation. I want to put disk images wherever I want because my root FS is small (often on ZVOLs, which are a big massive PITA to use with libvirt) without having to read a million different websites and man pages. I want it to work with my firewall configuration, which often changes due to the fact I have to do stuff with this machine, and I don't really care about 90% of the features of libvirt - I just want to run Windows every once in a while and I want my VMs to keep working without too much effort.
Thus, I run plain QEMU with the following configuration, which took me a solid 10 minutes to perfect:
- `nsbr0`, a bridge with a static IP configured using systemd-network;
- `qemu-bridge-helper` set-up in order to attach taps to nsbr0 automatically, using `-netdev bridge` (echo 'allow nsbr0' > /etc/qemu/bridge.conf);
- IP forwarding set to 1 and `masquerade` in nft.conf;
- OVMF downloaded from my distribution's repos;
- `spicy` to attach to a running VM;
- a 30 line script that spawns QEMU in a tmux. If I want to configure a new VM, I just copy paste this shell script and I put it in a directory together with a copy of OVMF_VARS.fd for the efivars.
Et voilà, I get VMs that just work, without having to wage any pointless battle against libvirt in order to get what I want. This is basically just a way to run KVM VMs that does not create a million firewall rules together with useless network interfaces, and doesn't force me to configure "storage pools" when the only thing I care about is running 2, 3 VMs max on my machine.
There's a reason why VMWare sells both ESXi and VMWare Workstation, and why people still love VirtualBox despite how shitty it is.
It's because more often than not something designed to run on servers is hardly be the best solution for a single user. The same features that make libvirt interesting on servers make it clumsy and overcomplicated on workstations, and the more I use it the more I realize it's true.
LXD recently added support for QEMU virtual machines in addition to its existing support for LXC containers, and it's almost completely replaced libvirt for me. It's great, pretty much just a matter of adding a `--vm` flag to normal `lxc launch` usage, e.g. `lxc launch images:debian/buster --vm`.
Yes, but the main reason I use VMs is to run different OSes. When I need to run other linux distros I normally use systemd-nspawn (or rarely LXC) and it's enough.
It's a type-2 hypervisor, so its performance is often quite worse than KVM. Also, its storage drivers are quite wonky, so I/O is often choppy and underwhelming. It has often a better desktop experience, though, except when you run Linux with VirGL. Also, virtualbox cannot emulate other platforms; for instance, I often run AArch64 operating systems in full emulation using QEMU on my x86_64 workstation.
Yeah I didn’t say ideal. But if you are on Win/Mac (there are a lot of those people) and need a Linux box without much fuss it’s a good go-to solution.
No, KVM (and FreeBSD's bhyve) are kernel modules that for all intents and purposes transform the host kernel into a type 1 hypervisor itself. The hypervisor is not, like in the case of VrtualBox or VMware workstation, running on top of the kernel, it is the kernel itself. The VMs spawned on KVM or bhyve have potentially full access to any hardware capabilities exposed to them. You can passthrough PCI devices to KVM VMs, for instance, using VFIO; you can also fully expose the host CPU allowing nested virtualization
A "true" type-2 hypervisor does not require any modification to the host kernel; at max it ships with a kernel module or two to help implementing things such as custom networking or to get a speed boost kernel side. For instance, it's trivial to share USBs with a guest on the fly with VirtualBox, because the VM is basically a process running on the host, so it can just tap into the USB stack and expose the device to the VM. This is not so simple with KVM.
Thanks for creating this, it looks fantastic, I just bought it! I love that it's a nice layer over QEMU / Apple's Hypervisor, so i know the performance will be reasonable. One bit of feedback you might consider: Most of the VMs in your gallery are for ARM64, and the x86 one is quite an old version of Ubuntu. Would you consider making a couple more modern x86 versions? My requirement is to have an x86 VM for deployments to Lambda, cross compiling Rust using Docker on M1 simply doesn't work yet and I'm hoping your app will do the trick :) Thanks!
I have the same requirement, but ended up building the final bundles to lambda in the cloud instead. However I am quite excited about the new container image support for lambda meaning I guess you do not have to compile for musl anymore.
Oh very interesting. I only need Vagrant support and then I can fully switch to M1 in the future. Looks like this could be the successor for Virtualbox on Mac M1.
Wonder if this also helps developing for x86 Docker too (which is a major pain point on M1 Mac).
Wondering if there could be a Metal -> Vulkan -> Zink -> OpenGL way of doing HW acceleration.
Or Metal -> Vulkan -> DXVK -> D3D for Windows.
AFAIK VirtualBox used some of Wine's D3D code to do D3D on Virtualbox.
The commercial virtualisation tools from Parallels and VMware support creating macOS guests on a macOS host by making a clean install, i guess from the macos recovery system/service?
Are you planning on macOS guests too? It's sometimes really nice to test a macOS application on an absolutely clean macOS install to ensure it's not dependent on any library on your macOS development machine.
Well it is great in many ways, but today I downloaded it and played a bit and found it hard to use compared with “directly using qemu cli” or vmware fusion for Mac free edition, but I am looking forward to its improvements in the future. Please do keep the good work ;-) thanks!
Thank you for making this available for download from the website too, and not making it app store exclusive. Do you have any plans to support macOS Mojave? What is needed to add GPU support in the future?
afaik there is nothing preventing this from being on the mac app store.
The iPhone app store is so strict because they really want to keep the platform locked down, with no possibility for 3rd party app stores. Although even then there's weird exceptions - Roblox seems to be allowed.
That cat is out of the bag on MacOS, though, and at least for now it seems Apple is keeping it that way - iPadOS is going to be their locked down work OS.
When the Debian install asks you what device to use, give it /dev/sda or /dev/vda. For some reason /dev/cdrom wasn't working for me either. I ended up switching the ISO's device to virtio and tried /dev/vda and it worked.
The ISO was on /dev/vda and the target disk image was on /dev/vdb. Just make sure you install the bootloader on the disk image. Then when I removed the ISO, everything was just fine.
Please try out the free download and only purchase it if you feel that you’ve already gotten value from it and would like to support the project. There is no difference in the binary for the free and paid version. It’s really hard to promise anything 5 years ahead as the intent is not to make money off of it.
It's understandable you may have missed some information, so to correct the record (and ensure other readers are aware), Apple highlighted virtualization in their announcement of M1, specifically highlighting Linux. It's been supported from the moment it was public.
Well WTF... I know 100% for a fact I saw this somewhere, that's why I didn't buy one............. can you think of what the article I read might have been referencing? It might have even been a huge thread on HN how developers/programmers were waiting a year+ to buy it until certain things were supported.
There were two common misunderstandings which were widely promulgated people whose confidence exceeded their actual knowledge:
1. The developer preview systems didn't support it, but that was because they used the older A12X chip. Lots of people spread that on social media as this not working on ARM at all.
2. Docker for Mac did not have ARM support at launch. Docker does have full ARM support so this was just a function of needing time to develop the desktop app (and not having access to the M1 hardware much ahead of anyone else) but some people panicked and portrayed this as something which would require a huge amount of work. There's a preview release out now: https://docs.docker.com/docker-for-mac/apple-m1/
I think maybe you saw a bunch of FUD? I’ve seen a ton on here.
There was also a lot of speculation that Apple would quash booting other OSes, and maybe that got into the mix?
FWIW I’m a lifelong Apple user and even if I had the budget I’d hold off a year on ARM Macs, because I’ve learned to avoid first revision models in general.
It’s FOSS :) other than that, Parallels only does virtualization and does it really well. UTM/QEMU does virtualization not-as-well-but-still-pretty-good in addition to emulation.
Do you mean the emulation performance on M1? In my experience Windows XP is “usable” and Windows 7 feels like a 10 year old desktop running with a spinning drive.
Technically the AArch64 instruction set and the memory system are the only parts that are 'hardware accelerated'. QEMU handles everything else in software.
$ nm /Applications/UTM.app/Contents/Frameworks/libqemu-aarch64-softmmu.utm.dylib | grep -i hv_
U _hv_vcpu_create
U _hv_vcpu_destroy
U _hv_vcpu_get_reg
U _hv_vcpu_get_sys_reg
U _hv_vcpu_run
U _hv_vcpu_set_pending_interrupt
U _hv_vcpu_set_reg
U _hv_vcpu_set_sys_reg
U _hv_vcpu_set_vtimer_mask
U _hv_vcpus_exit
U _hv_vm_create
U _hv_vm_map
U _hv_vm_protect
U _hv_vm_unmap
Be careful, I think you need something like 50-60GB of free space to run it. And the script does not check the free space beforehand.
After the script is run, I think the OS X VM will occupy at least 25GB.
PS: the graphics performance is ok, but not great. I could not manage to install the extension package of VirtualBox, so may be if you can install it, then the graphics performance of the virtualized OSX will improve.
AFAIK you can't, and there's no indication that will change. You may be able to virtualize a whole macOS install, though that's pretty difficult even on Apple hardware (source: I tried several times, and didn't get it to work well)
I think many open-source efforts are heading in the wrong direction. Let Apple make sure our software runs on their hardware, and let FOSS make sure that Apple software runs on FOSS platforms. Not the other way around.
Thats interesting. But it seems like the AltStore requires running a AltServer which is only available for Mac or Windows. It would be great if they provided a Linux build too.
It is kind of ironic though that I need to bring up a QEMU VM to run the AltStore binary which lets me run QEMU on my phone :D
This is what I use to run QEMU on my M1 Air. I had a few bumps setting it up but now it runs flawlessly. I SSH to it via VS Code and basically I have a fully functional Linux dev environment with MacOS as the UI, just the way I like it.
Honestly, I haven't been this excited about a new setup since Moore's Law was in effect :-)