Recently I read somewhere that writing long prose is easy but writing something succinct takes way longer. I think that translates to writing code as well.
I think that's where people swing back to DAMP, or at least the rule of 3.
I'm not trying to fractally compress my code. I'm trying to make it succinct. Those two things are very different. Now if I could only convince a particular coworker of that...
I've gotten to a place where I start with a sketch of how I think my code ought to be used and then work backward into the implementation.
So far I've gotten a lot of pushback from others on my team. And, what starts as "oh, you should just have to ____" gets really messy as the implementation takes shape.
I think so, yes. But I start "before" the test? I write a use-case. Almost a user story, but in code. Sometimes I add comments inline with the code to try and sort out what abstractions are useful.
OMG, YES! The JIT is here... and holy crap, if you haven't tried it yet... it's awesome (everything feels snappier). I'm particularly stoked for the receive optimizations and process aliases.
BEAM just gets better and better. It's a good time to be an Erlanger/Elixirist...
The delay is all in compiling exs files. It uses Kernel.ParallelCompiler to compile every .exs file, so it's very CPU/core dependent. On my weaker laptop, `mix test` takes nearly 10 seconds to just start.
I've looked into this in more details in the past. We've had success just writing our own test runner and avoiding exs files. But re-implementing things like running tests based on line number, or integrating with external tools (like excoveralls) has been a dealbreaker.
Unfortunately, it is a bit too large (and too late) for v1.12, but if loading times have been problematic for you, it would be awesome if you could try master out and let us know in the issues tracker (or in the commit) if you see any improvements.
Are you sure that you're not including application startup in that 10 second measurement? I've seen code bases where horde or some other clustering was enabled in the test env and causing a ~5 second delay on application startup.
From my recollection (interviewed OTP team on this stuff once) they don't really intend to go there either. They are a very small team comparatively and maintaining that kind of runtime optimization would likely be unwieldy. (https://devchat.tv/elixir-mix/emx-114-just-in-time-for-otp-2...)
It won't be anything like V8, entirely correct, but it brings some VM code to native performance, beating NIFs in some cases.
I think some RabbitMQ tests reported 30% increase in throughput which is pretty wild.
Awesome, thanks for linking that! I was wondering how these improvements would flow through to Elixir.
Looks like we get the JIT performance improvements for free.
And unrelated to the Erlang/OTP changes, Elixir 1.12 looks awesome. Totally small, but such an unexpected little quality of life improvement, Kernel.then/2 for pipelines, looks great. I love the core team's focus on the UX of the language.
I have just a few remaining complaints about the language at this point, and Kernel.tap/2 and Kernel.then/2 will solve two of them.
When Jose mentioned a few years back that Elixir the language was more or less "done" or at least stable, and that they would focus on ergonomics and UX going forward, I remember getting a little worried. But I’ve found myself agreeing more and more - there’s not much I miss in the language itself, and projects like Nx, LiveView and LiveBook have shown that it’s an excellent foundation to build very powerful and modern stuff on top of.
As someone who has been programming with Elixir for my day job for the past few years, I find this aspect of the language to be super pragmatic and productive. It's a nice feeling to not have to chase new language features and syntax and focus more on the problem at hand. In addition, I've never felt limited by the language given that the underlying constructs are so powerful (message passing, immutability, pattern matching, etc). Glad that Jose made the decision that he did.
Curious, do you use typespecs and dialyzer? If so, how do you find it?
Elixir checks pretty much every box I'd want in a language, but after dealing with nil in Ruby for years and having fun with TypeScript... I'm feeling more drawn to working with type systems.
I find working with nil in Elixir to be quite reasonable. One way in which elixir is different is that you have different operators for when it must be boolean, and nil is not tolerated for those operators (and vs &&). This coupled with the convention of using ? at the end of functions which emit boolean makes things easier.
The one thing I wish is that people stopped writing boolean functions with is_ in front (that is supposed to be only for guards, but not everyone follows that convention).
If you want type systems, you can probably work with Gleam, in the future, I imagine there could be great interop where you can just drop in a .gleam file in your elixir code base with zero hassle attached, and have parts of your code base that are completely type safe, and let Elixir handle all the risks of IO and other effects.
I'm not a huge fan of Dialyzer myself. I would put it strongly in the "much better than nothing" category rather than the "usable and useful type system" category. I always write specs for my functions and types, and while they sometimes catch bugs, they're not quite as expressive as I would like them to be.
I suppose you could write Elixir in a style that was more type safe by writing in a less polymorphic or recursive style, but the language does not lend itself well to it. Structs and maps are mostly fine, discriminated unions less so.
Answering from similar experience. I personally use them both, dialyzer as part of the Elixir Language Server and typespecs when I feel something needs more clarity and definition.
Depending on what itches your types scratch for you it might be enough, might not. I've never wanted more type system in my Elixir personally.
I generally verify types only at the boundaries of my application (or very critical modules) using norm[1].
Either you have a strict type system that does not have an "any" type (yes, I'm looking at you Typescript), or you have a flexible type system like Python/Erlang/Elixir and you do runtime type checking whenever it's needed.
I'm writing more Typescript code than I would in Javascript for almost no type safety benefits (but for documentation, it's awesome).
Dialyzer and typespecs are better than nothing but not with much because they can introduce a lot of friction along the way, for not much benefit.
As another poster said, it can catch the very occasional potential bug but to me at least it's rarely worth the hassle.
I miss static typing after I got back to Elixir from Rust but the BEAM will likely never be statically typed.
To that end, I found utilizing metaprogramming to generate normal and property tests to be much more productive use of my time, with a measurable impact to boot.
"When Jose mentioned a few years back that Elixir the language was more or less "done" or at least stable, and that they would focus on ergonomics and UX going forward"
That's awesome to hear. I think there's a lot of language communities in the 10+ year range (and look, that's right where Elixir is) could stand to have that recognition. I'm getting kind of saddened by the number of languages I see that are good languages that proceed to festoon themselves with so many new features that they become quite difficult to use.
Seems like there is less and less reason for not using Erlang for distributed coordinated compute broker tasks.
I have a stateless java backend servicing REST APIs.
These backends are load-balanced by nginx.
Now the time is approaching where I need to introduce some form of global state (that involves global caching, message passing, registering/discovery of known workers, periodic (cron-like tasks), etc).
I would much prefer a single tool to add to my backend stack (currently Java + postgres) to cover most of the above needs.
With the performance improvement + persistent_term [1] -- in current Erlang, I basically get:
- a light weight distributed K/V cache
- a ZeroMQ like on-wire messaging system (built into erlang)
- discovery/coordination
- a distributed compute grid
- a way to write my own routines within that compute grid, and have them exposed via Java interface to my existing backend.
I do not need 'fastest' possible performance or least memory consumption. I just need them to be 'reasonable', 'known' and 'controllable' (not to exceed some baseline).
Erlang is just looking better and better (and I prefer its syntax to Elixir, for some reason.).
Do consider Elixir for its metaprogramming (macros) and the goodies that come with them, namely fantastic libraries and frameworks like Ecto and Phoenix.
Speaking of error message. I love that Erlang gives the exact parameter values in the call stack. (I guess this is generally possible because immutable data structures are enforced in the whole language?) It saves a lot of time than just speculating with call stack only.
Okay, lots of people arguing about this, so I'm gonna reply to you, the top-most person in this sub-thread :) I think the reason that it's easy to argue is that you can be talking about similar but slightly different things.
When Rust was created doesn't really matter, exactly. For a very long time, errors looked something like this:
At some point, "improving the error messages" became a project goal, and Jonathan Turner decided to take this on. This is described in https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-com... . The post explicitly cites Elm as inspiration because Rust was directly inspired by Elm in this regard.
Yes, other languages may have started this trend earlier. Yes, maybe they influenced Elm, which influenced Rust. Yes, Rust may be "older" than Elm. But if you ask the people who began this effort, they will name Elm as the inspiration.
(And yeah, then you can try to argue about who has influenced the broader public, which is effectively impossible to prove, IMHO.)
Today that error looks like
error[E0425]: cannot find function `print_with_unicorns` in this scope
--> src/main.rs:2:5
|
2 | print_with_unicorns("hello?");
| ^^^^^^^^^^^^^^^^^^^ not found in this scope
incidentally, and there's also JSON output, and if you're not on a terminal, you get the line numbers like before... lots of things are improved. This particular error doesn't show off some of the nicer things. Esteban Küber has taken up where Jonathan left off, and has been doing amazing work.
Template errors with g++ vs clang++ are what made me abandon the gcc toolchain for my dev environment (I still build against both compilers in my CI/CD though).
Rust wasn't even a thing (not as hype and mature) at that time.
Rust is nice, but the hype train is toxic (and that's true for every language/technology).
GCC also copied clang's excessive diagnostics and I'm seriously thinking of making a fork just to strip them out. I doubt the developers would accept a patch that implements --stfu.
Yeah. It's funny how back in the day people used to make fun of excessively long compiler errors involving templates.
Now everyone seems to praise the compiler barfing three screenfuls of text and code and explaining the include hierarchy of my project and expanding all the macros and making bogus suggestions because I mistyped a variable name.
It's gotten to the point where locating the actual error message is literally more work than fixing the code. It doesn't help that this interferes with navigation in emacs and trying to jump to next error instead takes you to the next #include line gcc wants you to learn about.
People around here give Elm way too much credit for inventing things it didn't invent or wasn't even the first to ship useable versions of, but I think its fair to give Elm full credit for elevating the quality of error messages one can expect from a language or framework.
That said, the concepts in question are all much older. If you're ever bored, check out the "Influenced by" sections of the Wikipedia pages on programming languages. It's amazing how old so many of the "new" ideas really are.
That doesn’t preclude learning from younger langages in any way.
The trend of providing really helpful and valuable error messages (especially compilation) really started with Evan’s “compilers as assistants” and “compiler errors for humans” from 2015, although there had been forays into improvements to e.g. error localisation from clang.
I remember Ada always having very precise and helpful error messages like this one:
literal_string.adb:5:33: warning: wrong length for array of subtype of "Standard.String" defined at line 5
literal_string.adb:5:33: warning: "Constraint_Error" will be raised at run time
then when you run the app it behaves as advertised
There are many others and when I saw llvm improving error messages for C and C++ (which I saw happening before Rust was a thing) I always thought it was inspired by Ada and it's helpful messages like:
expected private type "<type name>" defined at ...; found type "<type name>" defined at ...
"<name>" is undefined (more references follow); possible misspelling of "<name>"
and the many others that were just there when I first tried Ada around 2008/2009.
True, Evan actually mentions Clang (noting that he’d met people who’d switched from gcc to clang due to the error messages) in “compiler errors for humans”.
I love it how good ideas just spread everywhere in open source. Everyone's life improves and usually people aren't obnoxious about it, which tends to happen in more politically charged topics (i.e. when there is a company pushing a narrative such as google,apple or microsoft)
Improved error messages is great. But one area specifically that is in desperate need of some love is the type errors from Dialyzer. They are rarely helpful to identify what is actually wrong in the code. Typically the error message will point to something several layers up or down in the call stack and prints out a huge blob of unreadable and unhelpful type signatures. Most of the time the best you can do is to simply compile and run the system to figure out what is wrong with the types. The obscurity of the type errors remind me of C++ template errors.
Part of the problem here is that dialyzer is a quite ... spaghetti codebase, that noone really want to fund that work and that noone really knows how this stuff work that is interested in doing that work.
There is a lot of research work to do and noone to pay for it. That said, if you know of a company interested to fund this work, i am interested and would love to hear more about it.
Slightly off-topic from someone who hasn't touched Erlang in almost 20 years: if my app is distributed with Erlang/Elixir, I kind of feel like my choice of database should be something with excellent horizontal scaling too. Can someone offer some insight into the trends in 3rd-party tools like databases for Erlang applications?
Erlang already comes with a distributed DBMS called Mnesia https://erlang.org/doc/man/mnesia.html , which under the hood uses ETS/DETS depending on the configuration. In distributed Erlang projects you'll find that or abstractions over it. Mne sia makes the most sense in mu opinion when you only have 1-2 relations or just need some kind of distributed cache.
In Elixir apps you'll frequently find the aforementioned Ecto "ORM", which has adapters to different DBMSs like MySQL, PostgreSQL, and even Mnesia.
The one sort-of thorny piece with Mnesia is querying the DB. The syntax for select is not super straightforward and involves writing matchspecs (http://erlang.org/doc/man/ets.html#match-specifications) which are non-trivial.
Mnesia lets you do a lot of cool things (in memory DB by default! Super fast!) but it also has some pitfalls (Doesn't write to disk by default! Lose all your data when you restart the BEAM!).
Generally, I think it's a fine system if you're willing to put in the time to optimize it.
Postgres? ;) In all seriousness though, I wouldn't really expect database needs for Erlang & Friends to be that different from other languages. My current employer has a vertically scaled AWS RDS Postgres as companion for an Elixir app and it's worked great. (Ecto in particular is an excellent ORM.) If your app truly needs zero downtime or ultra-low latency then there are other options.
I've had interesting exchanges with a guy who does work on CouchDB. That is built with Erlang and should scale quite well in what sounds like a resilient way. Haven't dug in.
Horizontal scaling of the DB seems like a problem in many architectures. For SQL I'd be looking at CockroachDB as it is Postgres-compatible and is built with scaling out in mind. Haven't tried it though. Most of my work hasn't needed that for the DB recently.
I love the notion of co-located processing and data, but most of the data stores dont quite embrace it. Well actually Riak does quite well if you're fine with a dynamo like kv store. The riak library is actually pretty nifty for distributed computations. But with Basho out of business it's harder to justify. Plus KV stores really aren't as usable as good ole sql. In that realm there's ActorDB, but I've not been able to figure out how to run it inside my own beam cluster rather than standalone.
HiPE support is getting wonky, I don't know if many people use it, and I'm not sure it supports the latest OTP versions.
The Erlang folks are looking for maintainer volunteers to continue working on HiPE support, they don't have the manpower right now to maintain it themselves.
I'm not sure if there's a point in the history where you can run with (this) JIT or with HiPE on the same commit. Which makes an apples to apples comparison difficult. If you compare HiPE on OTP 23 with JIT on OTP 24, you're also getting the large amount of other changes as well.
Both HiPE and this JIT have drastically different improvements depending on the specific code that's running, which makes it challenging to have a real world benchmark as well.
I feel like I saw graphs in one of the presentations, but I don't recall which, or if it was about the final iteration of the JIT.
I suspect HiPE would beat the JIT on tight loops, but the JIT wins in general because of the lack of switching cost between native and interpreted code.
Adoption of JIT on robust platform like BEAM is excellent news. Although i would wait for one or more release major releases till I see the significant improvement in computational aspect of the code.
So a net reduction of 60K lines of code? And yet functionality was added to the system as well. That's praiseworthy IMO.
Imagine if "we did more with less" infected much of software development today.