I think it's a very strong point that moving script code off the main thread can help achieve smooth UIs. No more GC pauses, no more slowdowns if the JS engine hits a snag, etc.
I think this is actually possible on the web as well. Someone could write a UI framework which runs JS in a Worker, and sends messages to the main thread, on which there is HTML and minimal JS to receive the messages and handle them.
I'm surprised this hasn't been done, or has it and I just haven't heard about it?
If you're worried about the overhead of transferring lots of messages from the Worker to the main thread, I think it can be pretty fast actually. I did an experiment with proxying WebGL that way, which is a fairly high-traffic API, with nice results,
For something rendering a UI, message passing overhead should be reasonable, especially if the framework is smart enough to only send over what changes (like React Native does).
You are speaking my language! I would love to see a prototype of this. I've been thinking about it since I saw React Native.
I wonder if there's even value in something like an asm.js-compiled layout system, which absolutely positions elements. One of the insights of React Native is that to build apps, we really only need a small subset of the web's layout algorithms. Only flexbox, really. So if we recognize this subset, and remove everything else, there main thread can focus on really smooth performance.
React Native even performs the flexbox layout on a 3rd thread, and tells the main thread where to absolutely position stuff.
Obviously we want to re-use as much of HTML as we can, but there might be some crazy ways to remove stuff from the main thread.
EDIT: It should be noted that Mercury, a React-inspired library (https://github.com/Raynos/mercury), has examples of running in a web worker. It is pretty cool, but I think it needs to be fleshed out more.
We explored this at a previous company I worked at. The conclusion at the time was that the message passing model was not yet efficient enough at the time because of serialization/deserialization which nullified any performance gains. Transferrable objects at the time were really new. If the support for shared memory has improved, it might now make sense.
The biggest problems with performance are:
- GC pauses
- triggering reflow
- marshalling data across interfaces (serialization/deserialization of native types into strings and back again).
That last one is an absolute killer when it comes to applying 3D matrix transforms. Basically, you're taking an array of floats, serializing them into a string to do an element.style property assignment, so that the browser can then take that stringified representation and convert it back into an array of floats that it can apply. Huge perf hit here.
I haven't found serialization to be anywhere near the bottleneck. How long does it take to serialize/deserialize a 16 element array? I could see how this would be an issue if you had hundreds of them per frame. But for most purposes an entire 16 element array is overkill and you could probably come up with a more compressed description of the transforms (scale, rotate, translate) and consider adding a pooling layer.
People attribute pauses to GC, when it's sometimes just the browser container environment acting up and there's nothing we can do about it - so it's easy to write it off as GC. Not sure about today, but 1.5 years ago (i = new Image(), i.src="http...") could block the JS thread for as much as 19ms. In a larger app, it's easy to write that off as GC, but (at least on iOS) the performance tooling is really lacking and it's tough to find the culprit.
Basically if you want to prevent reflowing you end up making everything a texture and that leads you to making many many CSS transform changes on every frame. 60fps is easy to hit on a modern desktop, but mobile performance requires more effort to achieve an effect that feels natural. The problem with the marshalling is that it is comparatively expensive and a blocking operation, which is not conducive to buttery smooth graphics performance on all platforms.
Me me! I'll be exploring this soon and I'm sure I'll reach out for help before I get too far. Could do a lot of stuff off thread, from data to layout calcs. My profile should have some info related to the hybrid stuff.
I'd like to see a github repo. You can hit me up on twitter, info in my profile. I think even just finding the right library to make using webworkers + npm + webpack would be great to start, because honestly its all the other stuff that slows down the thread. React can run on the main thread for the most part.
You basically need to change BackendIDOperations and ReactEventListener to send/receive messages vs talking to the real APIs. It's already set up in the codebase to do this (because it actually worked at one time).
Yes, in browsers, JS animations are still hard (I might even say impossible to do well for large apps with a lot going on). One anecdote:
I spent a month trying to build a 60fps scrolling list view in JS from scratch (JS driving all the animations). No matter how hard I tried, I couldn't make it work on the web. I believe it was image decoding interfering but it's difficult to say why. Then I took that same example, which by that point was optimized, and ported it to React Native in a few hours(where we have off-thread image decoding and timers I can trust) and the example was like butter.
Now of course, I had hand optimized the performance, but my point is that on the web, there was nothing I could do, and with React Native, there was something I could do to make animations smooth.
This is not to say you should be doing animations in JS, but if you want highly customizable interactions that can be immediately stopped by a touch processed in JS, you should try it out and see how it feels. For things that are "fire and forget", or platform specific, you should try using CoreAnimation/keyframes etc which protects the animation from your business logic.
How so? Which feature of the UITableView can not be replicated in JS? I'm struggle to think of one. Swipe left/right to remove rows in a table can be done with touch events.
I do mean within a mobile browser. We have a pretty large app inside Cordova, and I've yet to get stuck where there's something I can't recreate. Would be interesting to see an example where a mobile browser falls short.
Exactly. Customizing exactly how a header bar "sticks" to the top when it hits a certain point, cannot be done with any built in browser scroller. The same is true of any animation that should happen in sync with the scrolling itself (think like a parallax fade/scale animation etc).
What most apps need can be done by listening to touch events instead of scroll events. Eg hiding the top bar when user started scrolling down and so on.
I can't even think of an app that has a parallax effect. If your app is required to have complex animations then web tech is not the right tech choice in general imo
I am not a maintainer and I just tried it briefly so I am not 100% sure of the allegations they make on their front-page, but it seems like http://gridstylesheets.org are using Workers to run their layout algorithm (Cassowary Constraint Solver) and absolute-position elements (using 3D transforms).
> I think it's a very strong point that moving script code off the main thread can help achieve smooth UIs. No more GC pauses, no more slowdowns if the JS engine hits a snag, etc.
BeOS took UI multithreading farther than any other environment I've ever used. And it paid off: I still think it might be the case that BeOS on circa 2000 hardware was more consistently responsive than any environment I have used before or since.
Our main thread (if you can call it that) just does compositing and handles dispatching UI events. Script, layout, resource loading, and the rest run concurrently in background threads.
That's amazing. I hadn't even though about Servo in light of all of this.
In current browsers, JavaScript runs in the same thread as the UI, right? In Servo, how do you do interaction with the DOM like `clientHeight`? It just pauses the entire JS engine while it goes and talks to the UI thread?
Maybe we should be thinking of all of this in terms that map well to Servo's current architecture.
EDIT: What's really exciting about Servo is that hopefully the architecture's parallelism is really sound, even if certain properties of the current web restrain it. Then we can work on ways to remove those properties, and in Servo it's a simple switch to turn on more and more parallelism.
> In Servo, how do you do interaction with the DOM like `clientHeight`? It just pauses the entire JS engine while it goes and talks to the UI thread?
Yes, but we'd like to experiment with new APIs in the future like `getBoundingClientRectAsync()` that will allow pages to do that kind of thing asynchronously.
I'm also interested in Servo as well. With React Native, we had the liberty to restrict certain paradigms that don't map well to building a parallelizable pipeline of operations. So the JS thread can't synchronously query for layout, which happens in the next stage, and the layout stage can't synchronously block on reading main thread (UIKit etc) values.
By restricting what the programmer can do, it naturally formed a pipeline where layout can be performed in the second stage while JS is determining the next UI update and so on. I'm not sure the extent to which this benefits us right now, but even mobile devices are beginning to have four cores so it seems like a nice property to maintain as long as it doesn't cause problems.
Servo sounds cool, and I'm curious how it takes advantage of a similar architecture but while the programmer has free reign to "clog the pipeline" by synchronous queries on layout etc. I'm sure they've thought of it, I'd just like to hear what they came up with.
We did a bunch of experiments with running React in a web worker. Maybe we should revive them. Because it's React, it ends up being easy and is basically the same setup (serialized events etc) as what React Native does.
Trust me, we tried. But as long as image decoding blocks the main thread (Webkit), you are helpless to update the UI thread at a fast pace. We even tried decoding images in a web worker and sending the result back to the main thread - but decoding in JS is very slow so general application throughput suffers (not through fault of JS itself, but because JS lacks SIMD which image decoding algorithms use in order to decode images quickly).
But running React in a web worker is still a really great idea. There would just be a lot of little edge cases to handle (like when you click a link - you need to prevent default of navigation, then send that link click to the web worker to see if your app wants to prevent default, if not, force the redirect back on the main thread). Having tight control over text editing might be challenging as well. These are difficulties, but I suspect they are a fixed set of challenges that are worth the increased parallelism in an increasingly multicore world.
But are web workers currently The Solution To All Our Mobile App Needs? No, but they're a great tool that are highly underutilized by JS developers, and that React is uniquely positioned to take advantage of.
Event handlers are synchronous, and web workers are asynchronous. This means you have to decide whether you will prevent native behaviors before you have a chance to communicate with the web worker.
It's easy to be too aggressive. For example registering a `touchstart` on a DOM node would kill the native behavior and prevent scrolling, even if that's what you wanted. Of course, you could implement scrolling in JavaScript. That's a lot of work and you have to make sure the web worker doesn't block for more than 16ms so that the effect is smooth.
In my experience it's best to use native browser behaviors where possible. You get all sorts of benefits, like scrolling being executed on a separate thread, and don't have to rely on reimplementing everything from the ground up.
I've been thinking about this as well yesterday, but to be honest I'm not sure where the benefits are. In theory the browser is doing the rendering in a separate thread anyway, i.e. CSS layout and things like scrolling are handled by the browser. If you're doing a long algorithmic (CPU-intensive) computation in JS then you should probably do that in a WebWorker anyway. Otherwise I'm not sure the overhead is worth it, for example the dragging / touch events should be invoking such a tiny amount of javascript that you shouldn't see much benefit in moving that logic to a separate thread.
Would be interested to see real examples of where this is makes sense.
Exactly. I think moving everything but React into the web worker is enough. But that itself isn't easy as far as my limited knowledge, you need to have separate script files. Perhaps a webpack plugin that manages that for you?
Is playcanvas using emscriptem or is it written directly in js? nice project by the way. Also what does the worker do? does it have access to the canvas context or it just does some calculations and communicate with the main thread via serialized data.
One pattern you can do is send events to the server, server renders data asynchronously (maintains state for each client in a lightweight process), then ships result back to client via websocket binary frames.
You also decide how much rendering happens where. Can even do whole HTML elements on the server if you want.
Regarding moving script code off the main thread: I wrote a cooperative FRP engine that multiplexes signal graph update recalculation onto Web Workers. UI updates like virtual DOM would live on the main thread.
It is somewhat limited, but using message passing to the main thread, you can render a UI or even use WebGL or Web Audio or anything else. See the link for an example with WebGL.
Oh, I know. But it's rare that a webapp does things computational enough that it's worth the extra overhead - most lagginess is actually at the UI level, in my experience (and why virtual DOMs are so popular now).
I've seen Web Workers used in things like graphing libraries, though.
Someday you'd think they might just go ahead and add multi-threading, shared memory to Javascript? It is definitely a can of worms, but the current web worker situation is just not workable for many use cases.
WPF has a UI and a separate closed off rendering thread, which works quite well performance wise.
I think this is actually possible on the web as well. Someone could write a UI framework which runs JS in a Worker, and sends messages to the main thread, on which there is HTML and minimal JS to receive the messages and handle them.
I'm surprised this hasn't been done, or has it and I just haven't heard about it?
If you're worried about the overhead of transferring lots of messages from the Worker to the main thread, I think it can be pretty fast actually. I did an experiment with proxying WebGL that way, which is a fairly high-traffic API, with nice results,
https://blog.mozilla.org/research/2014/07/22/webgl-in-web-wo...
For something rendering a UI, message passing overhead should be reasonable, especially if the framework is smart enough to only send over what changes (like React Native does).