All the audio library and langauge parser are written in Rust and compiled to WebAssembly, running in AudioWorklet thread.
The control parameter is constantly shared from main thread to AudioWorklet thread with my customised ringbuf.js to avoid GC from JS in real-time audio:
Wow, I checked your website/desktop (https://dustinbrett.com/) again and tried to open your browser to make some music there. Apparently for Glicol you need to enable CORS. I think that's the only thing that stops the website to make sound now.
But my another project QuaverSeries, the precursor of Glicol, works like a charm in your browser:
https://quaverseries.web.app
When I saw "Browser" in the list with a Chromium icon (while I'm running Firefox), one of the sillier (yet fun) thoughts that crossed my mind is that you had somehow compiled Chromium to WASM, and you would be running a full-fledged Chromium instance inside Firefox.
I'm sure there are limitations with WASM that make this impractical to implement for the foreseeable future without a ton of work, but it would be fun to try out if it did exist.
It needs to become reality :D, on a more serious note maybe it'd be possible for this to be used in inceptions with a slightly older but capable Firefox before they implemented process separation?
It is possible to run `NetSurf` in BoxedWine, but BoxedWine doesn't currently have network support. Also v86 has network support via a WebSocket proxy, so you might be able to get a browser going in there.
I'd be curious about the overhead if the browser was running in kernel space. This would be surprisingly secure since browsers provide some of the most hardened sandboxing out there.
(In)famously, Windows NT put GDI into the kernel for performance reasons. Any application wanting to draw something would also have to pay for that context switch, which apparently was deemed not a problem back then. Have context switches become that much more expensive since then?
Is that so? I thought even different contexts within the same browser context were well isolated. For instance, isn't the reason arbitrary web pages can't access my Lastpass credentials is due to this sandboxing?
The browser security model relies heavily on protection from things like process isolation, ASLR, etc though at this point browsers also implement their own defense in depth mechanisms on top. for example, jsc maintains separate heaps for each data type - referred to as cages, i think - so that it's hard to perform type confusion attacks. but there are entire classes of attacks that would be viable against browsers if not for the tools they get from the kernel like ACLs, page protections, ASLR, control flow guard, etc.
When one part of the stack of protections fails, you can see things like persistent root on chromebooks where that one hole is used as a springboard to attack other vulnerabilities: https://news.ycombinator.com/item?id=15713103
This is one of the most impressive things I've seen lately built on top of WebAssembly.
I remember some years ago (10ish?) there was a push of Operating Systems in the browser, but they all mostly became unused long-term because the browser APIs where not as powerful then as they are now. I think those new APIs along with Wasm support changes completely that assumption.
I've always liked projects like this but for me personally this one takes the crown just for sheer responsiveness - I expect to see a few "Working on it" because it's a browser but you've built a really solid foundation here, great stuff mate.
In my experience most of these are responsive but don't have much "software". This one took the crown for being so feature rich, even including a browser. The only other project I can remember that does this is FriendOS.
Thank you very much! I am always trying to tighten up performance and use tricks I learn to make the UX better. I hope to keep growing it over a long time.
How does this all compare to the old and dead idea that you'd just run java applets from your browser? Is it believed that browsers are harder targets than JVMs?
Webassembly uses the same bytecode and runs on the same VM as javascript, making it much more stable and reliable than the embedded JVM was. It is already in every browser and doesn't need to be installed separately, and has a faster startup time. It can write to the same canvases as JS so it doesn't have to be a separate visual space. Also an obvious difference is that you can use a lot more languages to target webassembly.
Just as an idle muse, I wonder if that's true. Java's startup time on a 133MHz Pentium running on spinning rust is one thing, JS starting up on a multicore multigigahertz monster backed by an SSH is quite another. I wonder if the WASM startup really is any faster than Java in some absolute sense?
Kinda like how "Go has a fast compiler" if you compare it to modern compilers on modern workloads, but is beaten handily by Turbo Pascal... as long as you don't mind that the output of the Turbo Pascal compiler is a wee bit out of date.
WASM does not run on the same VM as javascript, or rather, that V8 happens to provide the facility for both, does not mean that much. They are effectively separate things. Now, if you mean to say there's some degree of interop via a chunk of memory between the two, then fine, but that's not particularly an advantage.
WASM is a bit different between browsers, it's not 'stable'.
That said, Applet/JVM are 'nowhere' these days and would require specific installation.
WASM is not 'faster to start' and it's not even 'fast' <-- this, and lack of clean interop is the worst thing about WASM. It's slow! If you combined the fact you have to go across an ugly 'bridge' between JS and WASM to do almost anything, it's probably not worth it for performance gains, and certainly comes with major dev. cost.
'Targeting more languages'? Really? Java, Swift, Python, Go - not really. You can target C++/C and Rust but only in very specific ways.
So the advantage for WASM would be 1) security for user, but in reality v8/JS is already secure 2) performance, but in most cases it's not true 3) 'hide your code' yes, this is nice for some algs. and 4) port your C++ algs to the browser like Photoshop/Autocad.
So far the reality is that it just doesn't fully add up.
I hate to be so blunt, but almost everything you said is partially or completely incorrect.
> WASM does not run on the same VM as javascript
Except it does in both Chrome and Firefox, which are the vast majority of VMs out there in this space.
> it's not 'stable'
Once again, completely untrue. Because it runs on the same VM, it's as stable as JavaScript, which is extremely stable at this point.
> WASM is not 'faster to start' and it's not even 'fast'
It's effectively instantaneous simply because it's using the same engine as JavaScript, which is already loaded. I'm not talking about particular app startup up time, as that is not an aspect of the engine. I'm also not talking about interop performance, as that is unrelated.
> Targeting more languages'? Really? ... not really
Once again very wrong. Because LLVM targets wasm, you've got many many languages that are supported. Some examples: C/C++, C#/. NET, Rust, Java, Python, Elixir, Go
> Because it runs on the same VM, it's as stable as JavaScript, which is extremely stable at this point.
Anecdotal: within a year, our medium-sized wasm application has "found" two separate JIT regressions (miscompiles resulting in breakage for users, most likely optimizer bugs), one in Chrome 91 and one in current iOS 15.4 Safari. For comparison, I don't think I have ever encountered a JS runtime/JIT bug so far.
To say that WASM and JS run in the same VM is utterly misleading the the point of being wrong.
They are not 'The Same VM'. They are 'Different VMs'. That in the case of Chrome and Firefox they happen to executed 'Side By Side' is relevant, but they are not the same vm.
"Because it runs on the same VM, it's as stable as JavaScript, which is extremely stable at this point"
This is a 'double down' on the fallacy that 'They are the same VM'.
"It's effectively instantaneous simply because it's using the same engine as JavaScript, which is already loaded."
First, so you're not going to count the startup time of the 'Engine' in the 'startup time'? But somehow that 'startup time' would be counted for example in JVM startup?
That wouldn't be a useful comparison, wouldn't it?
If you make a WASM-based app, you're going to have to throw in 'net startup time'.
Second, aside form the issue of 'Engine' startup time, you have to consider how quickly the code gets going. It is not 'instantly', nothing is instant in a VM, notably WASM has to 'compile' - not unlike Java bytecode. Java, JS and WASM all have different profiles here. JS is 'very fast' to start, because that's why it's optimized. Java is slower to start but after about 1000 iteratons of a block of code, 'runtime profiling' makes it fast. Often faster than humans could code in C++. WASM is slow to compile, and currently has little in the way of optimization.
Slow to start, slow to compute.
"Because LLVM targets wasm, you've got many many languages that are supported."
I'm guessing you're not a developer, and have never used WASM?
You do realize you can run JS in Java? Or Java in JS? Or Rust on Java? But that would mostly be stupid and pointless in any real world environment, which is why nobody does it?
Which is why nobody runs Java, C#, Python, Elixir in WASM for anything other than a mockup experiment.
So no, my points are all valid.
Today, WASM is the domain of Emscripten and maybe some Rust experiments, and that's that. It's slow, cumbersome, nobody is really invested in it. Maybe someday, but it's not on the horizon just yet.
You are going to have to provide evidence of this. Everything I've heard, every presentation I've seen, everything I'm finding online is that JS and WASM share the same VM implementation on major browsers. Both our arguments hinge on whether this is true or not.
Why don't the two of you go and check the Firefox and Chromium source code? That's all open source. You could prove your claim pointing us to the source. Even better, you and us, we could all learn something from that :-).
"SpiderMonkey currently has two WebAssembly compilers: The tier 1 baseline compiler (not shown below) and the tier 2 compiler using the IonMonkey JavaScript compiler's optimizations and register allocation."
Basically webassembly is the same core idea as jvm, but redone with all the lessons from the last three decades. A modern cross platform runtime for anything anywhere.
Having read the spec and partially implemented my own WASM VM, if it's goal was to be universal then it wasn't designed very well IMO.
Off the top of my head, it presumes that only 32 and 64 bit types exist, that everything is little-endian, and has parametric instructions[0] for some reason.
Pretty sure WASM was designed only for web browsers with any seriousness and any other usecases are just enthusiastic people trying to shoehorn it into other places because you can technically do it and it's the new exciting hotness.
[0] from memory I think it is only DROP, which makes the decision even more maddeningly silly.
All of the old Java (and ActiveX, ewww) was operating under the principle that you'd grant the applet more permissions than javascript, whereas WASM grants the same or fewer permissions!
But for a program like ImageMagic or 7Zip to have any utility, it must be reading and writing files somewhere. I'm just trying to figure out why I would download and run a wasm program to handle these tasks. Why would I trust it?
The HTML5 standard has included the ability to read files for longer than ECMA-262 has had modules and classes (which has already been quite some time—those appeared in ES2015).
Writing files is a different matter, but you can easily prompt the user with the browser-native download manager when your app has finished processing the input and the user indicates they want to save. This, too, has been available for a long time.
The Chrome team has introduced a new, experimental file access API, but it's not strictly necessary for any of this and probably gives up too much power[1]. Before that, there were the DOM Load and Save APIs (2002)[2], which never got implemented.
The future would look less grim if they chose instead to do something like reboot Google Gears, providing a supplemental component that implemented e.g. the remoteStorage protocol that just happened to use your local disk for its backing store—but in a way that's completely transparent to the web app, which shouldn't even be concerned with things like that. It would also translate much better for mobile devices, too, which may or may not have any notion of a desktop-like file system exposed to the user.
From 10,000 feet, the overall setup looks roughly the same. As in many situations, though, the devil is in the details.
There are a few key distinctions to make between the JVM approach and WASM's approach.
The first is that the JVM was a very high-level VM. It exposed a high-level typed bytecode with support for complex structured types, a sophisticated garbage collected heap model, and out-of-the-box system APIs with a huge surface area.
The second is related, in that the JVM's security model was very much mixed up with its type-safety guarantees. This lead to significant complexity in the implementation of its security model. There are places in the JVM spec where you literally have to walk up the java virtual stack, looking at security tokens along the way, to determine if a particular system call should pass through or not. All of this stemmed from a design perspective that very much derived from our ideas of best practices at the time.
The third is that the Java spec very much relied on load-time verification of strong interfaces across modules. This is one of the reasons for Java applets' famously poor startup times. When a single class-file was loaded, all of its dependencies must be loaded and its consumed interfaces type-checked, and this prompts type-validation of _those_ modules, transitively. Bytecode needed to be abstract-interpreted, the types of values on stack verified, and the types of interface methods that consumed those values verified.
The idea was that all of this up-front verification would allow for the later generation of safe optimized code without much effort. That idea didn't pan out as well as hoped.
I say none of this to knock Java. At the time of its conception, this truly was the industry's understanding of how to build a robust and secure virtual machine model. Java was truly a groundbreaking technology.
WASM benefits from design hindsight. It adopts a strategy of virtualizing the system at a very _low_ level. It's still strongly typed, but the number of types in the system are paltry (a few basic scalars). Its heap model is basically a massive array of scalar words. No garbage collection, no object graph, no weak pointers, complex finalization semantics, etc. etc. If your mind now wanders to the fact that WASM promises support for garbage collection to arrive soon, I'll point out that WASM's idea of garbage collection support amounts to little more than a few hooks to help track "heap pointers" on the stack, to allow for programs to implement their own custom collection approaches. The VM itself does not and will not specify an object graph model or garbage collections semantics over that.
WASM's security model is also much more coarse grained and simple. The default system-api surface area in WASM is the empty interface: a wasm module which is loaded and executed and provided no importable modules simply cannot affect anything except its own memory array. Any interaction with the system MUST go through an explicitly provided set of imports that the environment configures and passes in.
The VM itself doesn't specify any system API at all. This is developed separately as an independent spec. And here too, the surface area of the API is much smaller. It's modeled after the core Linux/POSIX-ish system APIs, and much easier to reason about.
The WASM bytecode is also far simpler. You still need to verify the bytecode, but the types are all scalars, and verifying the usage of module interfaces doesn't require any transitive checks across different modules, like Java does.
All this makes the entire system far simpler, easier to prove secure, and faster to execute.
Java was one of the first production attempts at making mobile virtual code a reality. It was built by some of the best minds of the time, and it is a fantastic piece of technology.
WASM is the result of two decades of hindsight knowledge, engineering experience, and practical experience in sandboxing a browser. It's no accident that while Java was an embedded+systems technology that got adapted to browsers, WASM is a technology that was born in browsers (as a dialect of Javascript!) and is moving out of that realm. WASM was forged in a crucible where downloading and executing random untrusted third party code was expected and routine (web pages). It was able to make better design choices because it was designed in a context where many of those design choices had been forced through sheer existential need.
WASM was also designed on an as-needed basis. ASM.JS was a toy project that turned out to be surprisingly adept at executing performant code. Long before WASM existed, ASM.JS was made performant enough to run Unreal Engine 4 on a browser. It was only after ASM.JS had proven itself again and again that WASM was derived as a formalization of it.
This bottom-up, engineering first approach is in stark contrast to the JVM's top-down design-first approach.
So what's the difference between the two? To summarize: no single thing, but instead a large collection of individual changes that work together as a whole to compose a system that is qualitatively more flexible, more performant, more secure, and easier to reason about.
And I'll emphasize again that this is not to knock the JVM or its designers. Those pioneers simply did not have access to the hindsight knowledge, the years and years of practical experience running untrusted code in a production sandbox, years of experience building JIT compilers (for javascript) and understanding what was important for fast execution (it wasn't load-time complexity for sure).
> Data stored in linear memory can overwrite adjacent objects, since bounds checking is performed at linear memory region granularity and is not context-sensitive
> Nevertheless, other classes of bugs are not obviated by the semantics of WebAssembly. Although attackers cannot perform direct code injection attacks, it is possible to hijack the control flow of a module using code reuse attacks against indirect calls. However, conventional return-oriented programming (ROP) attacks using short sequences of instructions (“gadgets”) are not possible in WebAssembly, because control-flow integrity ensures that call targets are valid functions declared at load time. Likewise, race conditions, such as time of check to time of use (TOCTOU) vulnerabilities, are possible in WebAssembly, since no execution or scheduling guarantees are provided beyond in-order execution and post-MVP atomic memory primitives :unicorn:. Similarly, side channel attacks can occur, such as timing attacks against modules. In the future, additional protections may be provided by runtimes or the toolchain, such as code diversification or memory randomization (similar to address space layout randomization (ASLR)), or bounded pointers (“fat” pointers).
> In WebAssembly, the execution semantics implicitly guarantee the safety of (1) through usage of explicit function section indexes, and (3) through a protected call stack. Additionally, the type signature of indirect function calls is already checked at runtime, effectively implementing coarse-grained type-based control-flow integrity for (2). All of this is achieved without explicit runtime instrumentation in the module. However, as discussed previously, this protection does not prevent code reuse attacks with function-level granularity against indirect calls.
> At the time of its conception, this truly was the industry's understanding of how to build a robust and secure virtual machine model. Java was truly a groundbreaking technology.
Maybe, but for the solve the problem they were actually tying to solve. Namely downloading some code from the internet and running it in a constrained environment they would have done much better to simple use Self.
Self had a far, far better implementation and in terms of code size it would not have been a show stopper.
And of course some of the people who did Self VW eventually did HotSpot.
WASM moved out of the browser very early, in spite of the name. Having an empty "system interface" as the default has made this easier, as has its being based on "memory safety without garbage collection" provided by suitably smart AOT compilers. It is an "embedded systems technology that can be adapted to browsers" as much or more than Java itself, and this is to its benefit.
Ya I think v3 or v4 of my project will eventually go more into this realm and recreating things at a deeper level. Then maybe it could really escape the browser one day.
Would you mind sharing how you abstract the applications? I looked in your repo but I'm not fluent in React.
I ask because I have a very barebones Application concept in my WebXR environment and am curious what sort of issues you ran into supporting many different types of apps.
My current system is kind of like a Entity-Component system, once you start considering 3D elements and dependency resolution. My hope is to one day be able to allow users to upload script bundles as apps into their environment, with the environment being just a minimum viable VR desktop metaphor.
At this point, I only have a teleconferencing system and an "annotated Google StreetView exploration app" as Applications. I recently added support for YouTube video streaming that I did not do in my App abstraction, but now think should probably be done that way.
For SharedArrayBuffer and CORS, I have some experience and it's not difficult to deal with.
You can check my project for music live coding in browsers:
https://glicol.org
All the audio library and langauge parser are written in Rust and compiled to WebAssembly, running in AudioWorklet thread.
The control parameter is constantly shared from main thread to AudioWorklet thread with my customised ringbuf.js to avoid GC from JS in real-time audio:
https://github.com/chaosprint/glicol
I use vite.js for development; for local dev server CORS, check:
https://github.com/vitejs/vite/issues/3909
For deployment CORS, e.g. Netlify, just edit the '_header' file as:
/*
in the 'public' folder.