Hacker News new | past | comments | ask | show | jobs | submit login
Lumen – Elixir and Erlang in the Browser (underjord.io)
264 points by lawik on Jan 24, 2020 | hide | past | favorite | 77 comments



What I find the most interesting about it is that it doesn't try to rework the BEAM to fit in the browser but rather does the much more ambitious work of trying to compile something fairly optimized that won't bloat the frontend.


Agreed, I was happy to see this.

I like Javascript, a lot, but I would very happily see a future where people have more choices about languages to use on the front-end. My only fear was that everything was going to morph into a kind of Java Applets 2.0, where everything would just be a giant VM on top of another VM.

For the most part, languages aren't doing that though, they're putting in the work to do proper integration with "native" browser systems like the DOM/CSS. I'm optimistic about where language efforts like Rust/Erlang/C#/C are headed, I think these projects are going to be really good for the web overall.


> everything would just be a giant VM on top of another VM

this concern is real which is why we didn't just run BEAM c source through a WASM compiler and call it a day. The implementation here is to optimize compilation to WASM and avoid as much overhead as possible. We have footprint requirements for the compiled asset sizes in mind too.


I bet that in about 5 years time the browser plugins would have had their revenge, unless the browser vendors pull the plug on WebAssembly.


Wasm is the most transformative tech to come along since Linux.


Not really, we already had WASM in 2011.

https://adobe-flash.github.io/crossbridge/

> Combine the power, familiarity, and high-performance of C/C++ with the unparalleled reach of the web. With CrossBridge you can bring your native C/C++ from consoles and PCs to over a billion people on the web – across browsers, with no additional install.

"Unreal Engine 3 Support for Adobe Flash Player"

https://www.youtube.com/watch?v=UQiUP2Hd60Y


WASM is very explicitly designed not to be another Flash Player.

My concern was that WASM would turn into another kind of Flash Player because devs would just cross-compile everything and spit out frames to a canvas instead of engaging with the web as a platform.

I'm happy that's not what Elixir, or Rust, or C++ are doing. Their approach means that the potential benefits of WASM even today are a lot higher than the potential benefits of Flash ever were. And future to plans to do things like share object and function references between WASM and Javascript push that potential even more.

If you're just concerned about taking a native C++ app and treating the web like a compile target, you've missed the point. The point is to take these languages and make them into first-class web citizens.


One thing I wasn't able to ascertain from the link: would this run your Erlang code with the semantics of a single-scheduler BEAM node (i.e. concurrently, but not in parallel); or does this use Web Workers to have multiple parallel schedulers?


Initially we’re not planning to support multiple schedulers via WebWorkers, but the runtime has support for multiple schedulers generally speaking. Since the compiler can target other platforms, we didn’t want to artificially limit all targets due to any particular target limitations. As far as Wasm is concerned though, once browser support for SharedArrayBuffer returns, then we’d be looking to take advantage of multiple threads.


Then, could a Lumen user do what pre-SMP Erlang did for parallelism, and

1. run multiple nodes on one machine (i.e. manually create Web Workers each with their own copy of Lumen); and then

2. write an erl_dist alternative-carrier protocol driver that uses worker.postMessage()?


Sure, I can't imagine any particular reason why that wouldn't work. That said, we don't have distribution implemented yet, since we are focused on a single node in the browser initially; so it depends on which comes first - SharedArrayBuffer being stabilized again in the browsers so we can enable multiple schedulers, or getting distribution implemented thoroughly enough to support the approach you mentioned as a workaround.


I think you can get more info from the video linked there. I don't recall the details but if it isn't Web Workers there might be something in Wasm. I'm pretty sure it's intended to be able to run multiple schedulers. But I'm not the one building it so not sure.


I think this project is a great idea but I think it's overly ambitious to be reasonably feasible in a near-term timeframe as it's basically talking about re-implementing 20+ years and thousands of man-hours put into BEAM dev into the browser.

It's also competing time-wise with simply "compiling Erlang itself to emscripten/WASM", since computing capability will still scale over time in speed and bandwidth and memory capacity to the point that simply "being able to get ALL of Elixir in the browser" will outvalue "being able to run some subset of Elixir in a single-threaded context only in the browser" (assuming performance is not an order of magnitude different).

I was just reading another HN article discussing the fact that OLAP is going away simply because computers have gotten enough memory and databases have gotten fast enough at querying columnar data stores that it's no longer necessary to add the "optimization complexity" of OLAP to get reasonable data analysis performance.

AMD has the Ryzen Threadripper now, 32 cores with 64 threads... If CPU development continues in that direction, this will be a BIG win for Elixir but only if it can take advantage of more than one thread.

That all said, I'm sure a naïve recompilation of Erlang BEAM to emscripten/WASM might not automatically take advantage of those threads properly out-of-the-box due to the abstraction layer, so there'd also have to be some work there to massage the scheduler to use "real" threads "on the metal".


Am I understanding this right that it's actually implementing the actor model just like Erlang? It seems to have processes and you can send them messages.

The threads seem really lightweight, but how would some typical frontend benefit from this?


> how would some typical frontend benefit from this

At ElixirConf I built a simple supervisor DOM tree model. When we are ready to implement it for real we're going to experiment with mapping the DOM to a supervisor. Each node being an element. This way events and rendering take place in each node in the supervisor tree. Rendering would be diff based and merged its way back up. Events would be captured as messages in the idiomatic Erlang/Elixir way.

This is all guess work right now and we don't yet know what will be practical vs what sounds good in theory.

However, we are dedicated to the idea that the programming model of the BEAM is one that complex client-side applications will benefit from over what JavaScript current offers.


That sounds pretty close to what Scenic does for OpenGL-rendered UI.

Scenic has some interesting properties in that if a specific component has a code problem and fails/crashes, the supervision tree will bring it back to life. I'm curious to see what similar things one could do with Lumen for the web.

Components are already a very popular level of abstraction with Vue and React, probably others. So maybe there are useful things to bring from Scenic.


Scenic is one area that I've looked to for inspiration on what a GUI can be like in Elixir


Oh wow sounds like a pretty fundamental overhaul of the current DOM model. Then again, various virtual DOM diff implementations have shown they can improve working with the DOM a lot.


Yeah, the undertaking would be quite extraordinary if they could get it working. Something tells me that this is going to take a few years to develop (preemptively giving a "bravo" if it could be done sooner ;) )


This sounds like an interesting experiment, do you have a repo or anything one could follow?


https://github.com/lumen/db_mon_demo

There is a db_mon-esque implementation within the source and ultimately this is where the demo will live but for now (and the purposes of that demo at ElixirConf) I went in a different direction


For me, the main benefit would be the ability to use a concurrency model other than the event loop.

In Elixir, I can reason about the code being run in one process as though it were synchronous, because it kind of is. But I can do that without blocking anything else, which is also pretty important.


It will run Erlang and Elixir. So anything you like or dislike about working with those languages should carry over. I find them delightful, mostly have experience with Elixir.

Asynchronous and event-driven code is already a fact in the browser so working with an ecosystem that is designed for that may prove a better experience.

I'm hopeful. Also, concurrency, staying of the render thread if possible. There's lots of fun to be had there.


Supervisors, Genservers etc. are OTP patterns, not specifically Erlang or Elixir language features.

Lumen seems to implement OTP patterns so that's why those things work on it.


Yeah, I'm aware of the distinction. But I honestly haven't settled on how to refer to it all as a whole. Do you have a preference?


I didn't mean to nitpick about the definitions, I just wanted to say Erlang/Elixir would both be well usable as languages even without the OTP abstractions and tooling.


That's a fair point :)


this is fantastic work. thank you and keep up the great work! The day JS becomes irrelevant will be joyous.


Interesting project... I wonder does it have same kind of processes preemption based on reductions like BEAM?


Is that the thing where after a number of reductions by a single process the BEAM will make sure another process gets its time in the sun? I believe so but I'm not familiar enough with the internals. They definitely spoke about reductions in the meeting I half-listened to.


Correct. It makes it so for(true) in other languages doesn't hijack the cpu and strive for the low latency goals of Erlang.

To be fair you can go about it by just filling up a process mail box with tons of messages.


Yes, Lumen has a scheduler and does preemption the same way the BEAM does.


Yes - preemptive multitasking is core to the BEAM process model.


I’d love to try using a wasm interpreter to run this on microprocessors! Is the project at a point that’d be feasible to test?


There is also nerves which specifically targets embedded applications: https://nerves-project.org/


microprocessor/embedded systems is a compilation target after the initial browser implementation. We're very aware of how awesome it would be to have the Erlang programming model without having to require an OS


Something like this thing? https://github.com/wasm3/wasm3

The sample I tried in this post is only an Erlang interpreter. The whole compile chain is not quite ready from what I gather. It should be feasible at later point. I don't see why not.


[flagged]




[flagged]


If we're going to limit project names in that way we might as well randomly choose letters


asdf-vm is the best-named project ever.


[flagged]


Not really. It's called branding and because's limited words that naturally have meaning - and the best words are likely to be coopted for projects in similar fields - then it's not difficult for people to search "lumen elixir". Likewise if the site you're on is contextual to Elixir then just searching "lumen" will find the result you're looking for.


> It's called branding and because's limited words that naturally have meaning - and the best words are likely to be coopted for projects in similar fields

"best" based on what criteria? Searchability is certainly a factor, no? Branding is pretty useless if people can't find your product.

> then it's not difficult for people to search "lumen elixir". Likewise if the site you're on is contextual to Elixir then just searching "lumen" will find the result you're looking for.

Right... until someone else creates another Elixir project called Lumen.

I think at some point you have to agree that naming collisions are a problem. The question isn't whether naming collisions are a problem, it's how far we want to subdivide fields before we consider collision to be a problem.

I'd argue that you're drastically underestimating this effect: having to type "Elixir" is definitely going to prevent some people from finding this project. For example, if I'm working with Elixir, and I need to do some frontend work, and someone on the internet suggests Lumen: I'll search "lumen", find the PHP framework, and probably guess that the person on the internet didn't really understand what I was trying to do, which is not an unreasonable guess given the widely-varying quality of answers from randos on the internet. Sure, I could search "lumen elixir", but how would I know to do that if I didn't already know what it was?

Contrast this with other ways of measuring the value of a name. Lumen is a word related to BEAM? Why is that important? Sounds cool? Again, why is that important? Pronounceable/memorable? Okay, but actually there are thousands of words that are unused, that are both those things.

You can't trot out "it's called branding" as if that proves any of these things are important: highly successful brands like "Prozac" or "Kleenex" or "Clorox" show that even being a real word isn't important.


"Right... until someone else creates another Elixir project called Lumen."

That's obviously a problem then - never argued it isn't ever a problem, your argument is a straw man whether you realize it or not.


Nope, I'm not accusing you of saying that isn't a problem. I'm merely demonstrating that a problem starts at some number of keywords. I understand that your argument is that the problem starts at two keywords (in this case) not one. Correct?

Would you care to respond to why you think users will know to type "exixir" when they search for "lumen", or why you think that "lumen" is a good marketing term? All you've done is misuderstood one small part of what I said: you haven't actually responded to the other 3/4 of what I said.


Why is "Apple" a good brand name? A brand is what you make of it.

It makes no sense for users to hear about "Elixir Lumen" from outside the Elixir crowd - so if they search for "lumen" and don't see the result, if they're paying attention and have minimal competency using search engines, then they'd type in "lumen elixir" to find correct result.


That’s a gigantic leap


Making one obvious mistake is correlated with making other obvious mistakes.


Haha! Oh dear.. I think adding "elixir" to any search is pretty much mandatory. Phoenix, Nerves, Scenic. All need some qualifiers. Ecto might stand on its own.


Not sure who came up with it first, but there's also Laravel Elixir and Laravel Mix which caused some confusion for me when I first heard about Elixir but didn't know it was in the Erlang ecosystem.


There is also a Python library called Elixir - naming is hard.


And a cryptocurrency, which makes for some fun questions sometimes on the elixir (lang) subreddit.


And in Poland there is interbank money transfer system called elixir so good luck finding job with elixir (lang) in description.


Better than Go!


Golang


Yes, I'm familiar with the hack that the community is expected to use every time they name the language in a searchable context. No better than prepending Elixir on a search if you ask me.


Yup. I've used Lumen a couple times over the past few years. Not sure why you would start a new project with that name, it's already well known and easily searchable.


Not sure why the OP's message was flagged, as it was a totally valid criticism...

But yeah, I just started a Lumen project last night for a quick API behind OAuth2. I sometimes have to append "php" or "laravel" to my queries, but that's to be expected considering I don't use Google.

I'm a big fan of Elixir and think Lumen's Elixir -> WebAssembly will be awesome, but I do wish they used another magic potion related name for it.


There have been many Lumens. Scott's and my hashmap-based Lisp was called that years earlier (https://news.ycombinator.com/item?id=17962433). When I contacted the dev of a previous project to ask if he minded us using the name, he replied that there were so many that it didn't matter.

Lumens come and go.


[flagged]


Separate worlds.. I'd never heard about Laravel's Lumen until the two comments here. I had seen that they had Mix and Elixir which seemed unfortunate. Lots of weird naming overlap going on specifically between these apparently.


Why? Just because Laravel came first and made an abandomware project?


What makes it "abandomware"? It has commits as of 19 days ago and a release three months ago. Pretty sure Otwell continues to update and is a stripped down version of a very active framework.


Abandonware? It's still actively supported/updated with each major Laravel release. We use it in our shop for some quick microservices.


Laravel Elixir is the abandonware. It was introduced in 5.0 then unceremoniously dropped in 5.4 being replaced by Laravel Mix.

Today it is essentially unmaintained: https://github.com/laravel/elixir


i mean, it does mention Elixir and Erlang though...


Uh, Lumen is already in use by the Laravel team for their micro-framework, maybe they're in time for a name change?


Elixir has had Mix since 2012, but Laravel made Laravel Mix in 2016 and hasn't changed it.


Unless Laravel team trademarked the name and are ready to go to court to defend it, I don't see why.


Mandatory word of warning [1]. Don't use wasm magic like this in production until its ready.

[1] https://www.reddit.com/r/csharp/comments/ea5dnb/the_nightmar...


That guy is using server side blazer which doesn't use wasm. And his experiences aren't normal.


But Lumen, if combined with Phoenix LiveView, is basically Blazor.


specifically which part of the comment touched on an issue you would expect to translate to lumen?


or just ignore the warning. Go for it, deploy it to production.


this is just FUD spreading. Having a bad experience with Blazor doesn't translate to anything on the BEAM.


So you write it in elixer and web assembly converts it to Js that browsers understand. Cool. Just write it in JS and it will probably be faster.


That's not accurate. WebAssembly is a different beast than JS. As far as I've understood it is executed differently and can do some different things. I don't see any reason why it couldn't be optimized more than JS either, though that's bound to take some time considering the amount of tuning that's gone into JS engines.

More accurately, you write Elixir, it compiles down to Wasm. Wasm is run by your browser as a more machine-friendly format than JS. It may well be faster.


That’s not how WebAssembly works, at all, it is handled directly by browser JIT compilers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: