The Azure Upstream team has been working on a really fast hypervisor library written in Rust for the past three years. It does less than you'd conventionally do with hypervisors, but in turn it can start VMs around 2 orders of magnitude faster (around 1-2ms/VM).
I think this is really cool, and the library was just released on GitHub for anyone to try. I’m happy I got to help them write their announcement post — and I figured this might be interesting for folks here!
Do you think requiring to use their packages is too limiting for widespread usage? Seems like you're forced to use Rust or C atm.
This seems like a limitation that sits in a somewhat unusable place: For something simple and platform-specific (e.g. a HTTP transform) we can just use JS where the boot time perf makes up for the execution perf, and for something more serious like a full-fledged API 120ms should be more than enough time (and we can preemtively scale as long as we're not already at 0)
The way to think about Hyperlight is as a security substrate intended to host application runtimes. You’re right that the Hyperlight API only supports C and Rust today — but you can use that to for example load Python or JS runtimes which can then execute those languages natively.
But we might be able to do even better than that by leveraging Wasm Components [1] and WASI 0.2 [2]. Using a VM guest based on Wasmtime, suddenly it becomes possible to run functions written in any language that can compile to Wasm Components — all using standard tooling and interfaces.
I believe the team has a prototype VM guest based on Wasmtime working, but they still needed a little more time before it’s ready to be published. Stay tuned for future announcements?
Looks like my TinyKVM project, except it runs specialized programs instead of regular ELFs? TinyKVM also runs functions, with a fast execution timeout. I proved that without I/O you can essentially run KVM programs with native performance, and sometimes more due to automatic hugepages. I measured LLMs to run at 99.7% native speed using eg. Mistral 7B. For example, the STREAM memory benchmark doesn't use hugepages by default, and so the terminal version runs slower than the TinyKVM version due to hugepage-tables, but of course runs at the same speed once you modify the benchmark to use the same advantage. However, it does require modifying the program.
I also implemented VM resets using page-table rewrites and CoW memory sharing, so that no memory is shared across different requests. This can be implemented as tail-latency in a cache.
I ended up adding support for most languages. All the systems languages, Go, v8, LuaJit etc. Go was by far the most annoying to support as it uses signals.
I don't have access to that paper - and when I looked for TinyKVM all I found was the rpi-based project that uses the other definition of KVM. Is your project online somewhere? Or is it proprietary?
Because the Linux kernel is incredibly complicated and shouldn't be trusted as a strong security boundary. A simple hypervisor likes has far fewer vulnerabilities so is an easier to trust boundary. They are in very different security tiers.
I would summarize as containers are good for mostly trusted isolation (teams within a company, purchased software) VMs are good for general untrusted software (different tenants in a cloud provider) and separate physical hardware is for scenarios where attacks are likely (military, known malicious code). Of course use cases are very fuzzy, but I wouldn't run fully untrusted code in the same kernel as anything of value.
> These micro VMs operate without a kernel or operating system, keeping overhead low. Instead, guests are built specifically for Hyperlight using the Hyperlight Guest library, which provides a controlled set of APIs that facilitate interaction between host and guest
Sounds like this is closer to a chroot/unikernel than a "micro VM" - a slightly more firewalled chroot without most of the os libs, or a unikernel without the kernel. Pretty sure it's not a "virtual machine" though.
Only pointing this out because these sorts of containers/unikernels/vms exist on a spectrum, and each type carries its own strengths and limitations; calling this by the wrong name associates it with the wrong set of tradeoffs.
Heh. Going by that delineation we end up with very VM-ish containers and (now) very container-ish VMs. Though this seems like it's even more stripped down than a unikernel - which would also be a "VM" here.
Chroot is a real security boundary as long as you use it properly. That said, namespaces on Linux are much superior at this point, so I can only recommend using `chroot` for POSIX compliance.
chroot is great for all sorts of things, but they're not security-related.
A lot of tools expect to do things to "your system" at absolute paths — chroot lets those tools operate against an explicitly wired-up semi-virtualized simulacra of your system, designed to pass through just the parts of those operations you want to your real host, while routing the rest of the effects into a "rootfs in a can", that you're either building up, or will immediately throw away.
Think: debootstrap; or pivot-root; or mounting your rootfs to fix your GRUB config and re-run update-grub from your initramfs rescue shell.
> each function request to have its own hypervisor for protection.
They are talking about isolating serverless functions, not host program functions. In that sense, it is exactly what Firecracker does for lambda functions
Firecracker boots up a runtime that has a full blown operating system in it - lambda just happens to call a known program with a known function. In that sense sure it provides similar functionality but it's really quite different. That's not what fly uses firecracker for, for instance.
Qemu/firecracker are in the same space - this is different.
These are most definitely in a different boat as you embed the guest functions inside the host program and then you register those functions. Taken from the readme:
> The host can call functions implemented and exposed by the guest (known as guest functions).
> Once running, the guest can call functions implemented and exposed by the host (known as host functions).
This is more in the 'safe plugin' type of space. As with most things in this space - the best way to learn about them is to simply try it out.
I believe the team has a working build of JerryScript [1] to test out the C bindings, but I’m not sure that will be released.
My understanding is that work on the Wasmtime VM guest is ongoing, which will enable Hyperlight to run the StarlingMonkey engine [2]. This is a WebAssembly build of Firefox’s SpiderMonkey engine which was donated by Fastly to the Bytecode Alliance.
That said though, I agree it would be great to see runtimes like V8 and JSC run directly on Hyperlight. There are good reasons why people might prefer those over StarlingMonkey (compat comes to mind), and it would be neat to see how much faster they could start compared to conventional VM deployments.
i wondered how it worked in rust but the guest entrypoint>init>main is wrapped in unsafeblock as is a lot of other low level operations it does. interesting stuff
I think this is really cool, and the library was just released on GitHub for anyone to try. I’m happy I got to help them write their announcement post — and I figured this might be interesting for folks here!
reply