Prior to Clojure I feel that I didn't really know how to do things well.
In the context of the domain I have the most experience with, web applications, I'd summarize the bulk of programming as defining data, data transformations and shoving data through APIs. Once you get the hang of it, Clojure makes it trivial to transform datastructures from A to B. This has enormously expanded the kinds of problems I can solve.
When Rich says
"It was impossible to
avoid the sinking feeling that I had been “doing it wrong” by using C++/Java/C# my whole career."
I feel somehow that this is the case for the majority of people but they don't realize it yet because their experience is the most popular trendy language or framework. I've seen many examples of libraries in different languages having enormous amounts of commits and hundreds issues for problems that are trivial to solve in Clojure.
I was in the same boat, constantly grabbing for frameworks and if one wasn't available that made my task simple, would struggle trying to bend the frameworks or language I was using to come up with a solution.
I'm not a language geek and I don't write code outside my work for fun. I want to spend the least amount of time possible in front of the computer and sleep knowing my web applications won't topple over. Clojure has fit me so well that I don't think I would have accomplished what I have in other languages.
Could you elaborate on a problem that illustrates this property of Clojure? This sounds awesome, but I have a hard time understanding what you're getting at without knowledge of Clojure.
1. core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation.
2. The Expression Problem: One of the core challenges in language design is designing it so that (simplifying a bit) you can transparently add support for new function methods to objects designed by a third party. Clojure makes this easy https://www.infoq.com/presentations/Clojure-Expression-Probl...
3. Lots of languages have attempted to allow you to write a single library that you can use both on the client and the server, having your code transpiled to different languages in each case. However, this type of feature is rarely used in production, because there are usually lots of headaches involved, with many limitations. However, for clojure developers it is perfectly normal (and usually expected) that all code is written in cljc files & so that it can be used on both the client (transpiled to javascript) and the server (transpiled to java). It is easy to do this, even for cooperatively multithreaded code.
I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language, for example, you cannot nest core.async functions inside other functions as they must be directly visible by the go macro, which cannot see jnside function calls. I’ve also had errors where the stack trace did not reference my code files at all, because it happened inside some core.async setup (IIRC it was a variable that was meant to be a channel but was nil, inside a sub or mix or something, ie I connected two core.async things together, it created go blocks internally, the exception happened inside that and therefore never referenced any if my source files, since the exceotion happened in an asynchronous context after my code ran).This was extremely painful to figure out.
Neither of these issues can be solved as long as core.async is implemented as a macro. However, it is extremely cool how far it was able to be taken without changing the language!
> I use core.async a lot too and really like it, however, I do feel that it suffers from being a macro instead of part of the language,
Agreed. The core abstraction of a go block is essentially a user-space thread. Because macros are limited to locally analyzing/rewriting code (and limitations of the JVM), go blocks are super limited in scope. As you point out, one call to a function in a go block, and you can't switch out of that go block during the function's execution. And functions calls are very common in Clojure code and can easily be hidden behind macros that obscure the details of the control flow.
There are 'core.async'y ways around all this, but the net effect is that 'core.async' imposes a very distinct style of writing (and one that tends to be contagious). Python's 'async/await' is contagious also, but because it's more tightly integrated into the runtime/compiler, it doesn't feel nearly as restrictive. (at least to me).
I don't think it makes logical sense for there to be a "core.async function". Instead, you have each function return a channel with it's own go macro, then have the parent function consume data from the channels.
Sure, that’s a solution, but its necessary only because of a limitation that only exists because its a macro that needs to look into the code to rewrite it. It also adds extra (cognitive) overhead that such functions are always themselves asynchronous and the return value are channels - they can never just return a raw value - which limits what you can do with them or how you can call them or pass them elsewhere.
I mean, yes, usually it isn’t a problem at all, but it IS a limitation of core.async that wouldn’t need to exist if it had tighter integration into the language.
> and think it is better than golang's implementation.
How is core.async better than the golang implementation. In golang, all network I/O you do is automatically asynchronous, so your goroutines won't block a thread on network I/O. That is simply something core.async can't do.
`core.async` does M:N scheduling where many go-blocks are scheduled across a smaller number of heavy-weight threads. `core.async` also has channels and select. It pretty much works the same as in go.
However, since the default in JVM is heavy threads and blocking behaviour, extra care needs to be taken to avoid blocking a thread that a go-block is executing on (as that would also block many other go-blocks).
So it's definitely easier in Go, since you don't need to juggle these two contexts.
JVM has NIO, which provides non-blocking IO. You can use this with core.aysnc's channels to resume a go-block. You have all the pieces you need to write a go-scale clojure service.
Having to pay ridiculously careful attention to what code blocks across libraries and function boundaries is something that golang completely takes away the need for - because it has a true cooperating scheduler built into the runtime at the boundary of every syscall.
Decision making is in hands of the Go runtime. You can write in standard dumb sync fashion at 3am in the night within a go-routine making all the Go library calls you wish to make without needing to scan code with a microscope to see what blocks and what doesn't.
core.async is a half-baked error prone solution. People will make mistakes with it. Until the JVM has native support for fibers, continuations and a compile mechanism to classify all legacy blocking calls as unsafe, core.async will continue to be hobbled.
M:N thread distribution is exactly what the go _scheduler_ offers. It's a work stealing scheduler like Java's ForkJoinPool.
The benefit lies entirely in that there's no other way to do concurrency. Which is great, as it's less error prone. But if you were to introduce native threads in Go you'd have the exact same problems as you'd have with core.async. So it's not like core.async (or async/await in other languages for that matter) is half-baked or badly implemented, but it won't automatically remove a part of the Java runtime that has existed since the beginning.
In a similar vain, Rust's actix and Java's Akka are not half-baked or bad implementations of the actor system, but it is more error-prone to use compared to languages built entirely around the actor paradigm, like Erlang and Pony.
So no, core.async doesn't magically give you non-blocking concurrency, but it does enable it. And that is a much better option than re-writing your entire codebase in Go.
> core.async is a half-baked error prone solution. People will make mistakes with it. Until the JVM has native support for fibers, continuations and a compile mechanism to classify all legacy blocking calls as unsafe, core.async will continue to be hobbled.
With Loom it won't be necessary to add a compile mechanism to classify legacy blocking calls, as there won't be any. This is why Loom is taking so long, the entire runtime is being re-written to not block if IO is performed in a virtual thread, but work as before if a full thread is being used.
And when you can simply create virtual threads instead of native ones, there's really not much value in core.async anymore (well, ClojureScript will still benefit) as Clojure already has great primitives for working with threads.
Yes, however, the Loom project is still some years away and the Java standard library will need to be extensively revamped to support this. I see this easily a decade in the future.
They have been rewamping the Java standard library slowly and silently over the last couple of years in anticipation for Loom. With Java 14 they re-wrote the tcp socket class. In Java 15 they're re-implementing the udp socket class. These re-implementations are being done to support loom seamlessly in a future update.
They also collaborated with IntelliJ to find bugs related to debugging Java apps, to make sure Loom doesn't break anything there.
Loom is well underway. I wouldn't be surprised if we saw Loom in Java 19 or 20. So in 2-3 years.
The next LTS version of Java is planneed to to be Java 21 (fall 2021, numbers a coincidence) and my presumption is they are trying to get it into that.
That is true, but my impression is that the original syntax was cumbersome and that golang made major advancements on that front (though I'm no expert and haven't studied the original papers)
I'm no expert myself and wasn't trying to call you out or anything. (I've only read that paper once or twice before a Papers We Love event I attended.) I think it's a common misconception, though, and thought people might find the history interesting.
This stack overflow answer has a very fascinating explanation of the history of the evolution of CSP from Hoare's original paper, to the work done on it with Occam up onto Go, I recommend reading it if you are interested in this stuff: https://stackoverflow.com/a/32696464/172272
Spec is pretty close to being a library for static typing; Clojure core being dynamically typed isn't really a blocker.
If you really want static typing Clojure can probably do it. The community support isn't going to be as solid as for something like Haskell. But I'm pretty sure spec offers stronger control of the useful parts of a type system than something like, eg, C. Except the control over RAM; that isn't a Clojure competence.
> So, the expression problem is the problem of solving another, unnamed problem, while satisfying the constraints of static typing? It seems a not very useful term then, and implies that every problem will need two names - the name for the actual problem and the name for the problem of solving that problem while satisfying type system constraints. Bleh. -- Rich Hickey
Solving the problem doesn't require static typing.
tl;dr on the above: The Expression Problem was originally expressly formulated with maintaining a Haskellian level of static type checking as one of its core requirements.
Saying you've solved it, when your solution involves on dynamic typing, or even casts in an otherwise static language, is arguably akin to kicking the ball across the center field line and then claiming you've scored a goal.
Actually, clojure compiles down to java classes, which are statically typed. Existing java types can be extended with clojure protocols. The link below talks about this, specifically about the expression problem.
However, I learnt something. I didn't know that the definition of the expression problem required static typing. I prefer to think of it as adding static types to multiple dispatch.
You missed a detail: or even casts in an otherwise static language.
Clojure does compile down to Java classes, but Java is only a partially statically typed language. It also permits run-time casting, which, in a language that even tries to be strongly typed, means run-time (to wit: dynamic) type checking. And Clojure relies heavily on that.
That's why I invoked Haskell. It's an example of a language that is truly statically typed, in that it doesn't (generally) permit any run-time type conversions. ALL type checks must be done statically.
Pretty much. Not entirely unlike how one can't implement the Y combinator in a statically typed language, but it's trivial to do so in a dynamic language.
“I know of no widely-used language that solves The Expression Problem
while satisfying the constraints of independent compilation and static
typing” - kind of implies that the problem is different from whether or not the language is statically typed, doesn’t it?
I don’t think you should be writing off Clojure’s multimethods too quickly - they are quite nice, and I haven’t seen anything like them in other dynamic languages like Python.
You've cherry-picked that sentence, though. Given the entire context of the article, I would argue that one should interpret that as a case of unclear writing, and not something that is intended to directly contradict the definition given in the second sentence: "The goal is
to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts)." (emphasis mine)
I’m not much of a clojure fan myself, but I think the observation is that by relaxing that last constraint of static typing (and given the other appropriate tools), that while you can’t solve The Expression Problem itself, it’s a bit easier to solve the real world problem that happened to manifest itself as the expression problem in your code.
By the way, I was also enticed by multi-methods in this area, but I ultimately don’t think multiple dispatch is necessary.
Yes, absolutely, but doing so would be moving the goalpost right out of the stadium. The expression problem isn't supposed to describe a challenge for people implementing line-of-business applications. It's a problem for programming language researchers.
>core.async: So the guys who built golang did it because they had this cool idea for creating coroutines using channels. However, since it required very low-level functionality to be part of the core of the programming language, they thought they'd have to design a brand new language to implement their idea. However, shortly after golang was released, Rich and some other clojure folks implemented the same feature in clojure as a totally ordinary external library, proving that the core of clojure was general enough to support this. And it wasn't just a gimmick: I use core.async every day and think it is better than golang's implementation.
It is very limited -- it was done by a single university student as his MS thesis -- but luaproc had added this functionality to Lua before Go even existed:
It’s available for unix systems as well in the Plan9Port, and that predates Go by a bit.
I’m not sure what any of this has to do with Clojure though other than to say “look we have Go-like concurrency without starting over”. It’s been shown that you don’t have to start over to add features to other languages as well.
core.async in Clojure is not equivalent to Go's asynchronous support. Even worse, core.async is the simplest & least-performant way to implement asynchronicity: It's just using an internal thread pool, meaning that go blocks in Clojure are not at all like goroutines in Go since they are state-machines produced by a macro that can block the entire thread when doing blocking calls. So, one has to manually "park" them by using core.async channels.
Summarizing: core.async is macro sugar on top of java.util.concurrent.
Go has actual lightweight processes that are scheduled on top of OS threads. Big difference in both design and runtime performance. Erlang too has m:n threading and offers a simpler and safer programming model (each Erlang process has its own heap).
Re "core.async is macro sugar on top of java.util.concurrent" - yes, but also no. Java does not (yet) have a way to spin light-weight processes that can park without consuming a thread OR a way to select/alt over multiple parked ops, so being able to do that in Clojure is actually novel (and enabled by having macros that can rewrite code as a state machine).
There have been a few projects that did this in the past on Java using runtime bytecode modification but none of them are widely used afaik.
This will change once Project Loom lands in Java and provides m:n fibers.
Life is too short to waste on Java? Yes, some of us really find Java that bad.
Plenty of other languages have solved the same problems in much better ways (as demonstrated by this very discussion re: asynchronicity). I'll pick Go over Java, Erlang over Go and wouldn't pick Java (incl languages running on top of) for anything today.
I usually find immutability to be more important than this particular detail. Because of this my preference is usually Elixir/Erlang, followed closely by Clojure (for the potential interop with a huge ecosystem), with languages like Go and Java fairly close to eachother but pretty far down list.
I would love it if Clojure on BEAM were more mainstream.
The business problem still can include things like "concurrency", which is just -painful- in Java, especially if I/O bound.
I much prefer languages that make easy stuff harder, but hard stuff easier, than languages like Java that make easy stuff easier, and hard stuff harder.
The Java ecosystem has solved many business problems, for me and plenty of other people I personally know, both in small startups and in large companies. Just because you are personally prejudiced against Java for some reason, doesn't mean that its not a good ecosystem, because it is. I don't like Java the language (and certainly not the crazy frameworks), although it has improved, but the ecosystem, especially from Clojure, is fantastic.
Sure, sometimes a simple, crude runtime will do, but if you need really good performance and low-overhead deep observability, you might want to be saddled with something that's more state-of-the-art.
Go's concurrency primitives are a less expressive version of concurrentML, which imo has been the nicest way of writing concurrent (but for a long time not parallel) applications the last 30 years.
Comparing concurrent systems is hard. I would suggest implementing selecting with negative acknowledgement (nack). If I can do that nicely I know I will probably like the system.
I am confused. If I remember correctly, go's runtime uses a thread pool as well. It's not like go somehow invented a runtime that does not rely on OS threads. So in the end of the day, what difference does it make?
If you ask me, java.util.concurrent is a much better and battle tested concurrent infrastructure than the simplistic implementation in go, because java.util.concurrent is actually written by the world's foremost concurrency expert, Doug Lee. I will take that over someone else's hand rolling concurrent runtime any day. God knows how many bugs there are in your home grown concurrency code.
It makes a huge difference. You can't use blocking calls in core.async, since they'll block the entire thread. There is no switching when you do a blocking call, the thread your code runs on will be out of commission until the blocking call returns. So this means that blocking APIs are out of the question and this translates to you not being able to write linear, straightforward code. You have to use non-blocking APIs and callbacks, and rely on core.async channels to force switching between blocks (core.async doesn't have lightweight processes). This is a total nightmare to debug and forces you to bend your programming logic to support this non-optimal programming model.
Constrast with Go goroutines or Erlang processes where you write linear, synchronous code that executes as part of lightweight processes. The built-in scheduler will automatically schedule these processes on real OS threads, and switch between them when they block (either through channels or blocking calls). OS thread utilization is far more effective, debugging is a piece of cake and the code that you write is easy to read and understand.
I'm bypassing your "world's foremost concurrency expert" comments. I suggest you use the concurrency offered by Go and Erlang by spending a few hours to do tasks you've done with core.async. You will learn something and you will see how Clojure falls far short of the optimal.
Hum, what you're saying seems to contradict a lot of what I've read. We're talking about IO here. So something has to block unless we're using non-blocking IO. I think what you mean is that Go will detect the block, and will yield your execution, and manage the IO in a non-blocking way automatically for you?
Under the hood, go still relies on non-blocking IO, epoll and all. And for disk and other serial IO I think it actually allocates real blocking OS threads as well.
I agree, the automatic management is convenient, but I fail to see the difference from core.async and the direct use of async IO or blocking OS threads when that's not available like for disk IO.
Go scheduler is sophisticated and it indeed intercepts all blocking system calls.
Essentially Go runtime will run a different goroutine, if current goroutine is blocked on: blocking syscall (for example opening a file), network input, channel operations, primitives in the sync package.
I would always prefer the runtime to do this job instead of having to scan code under a microscope across functions, libraries and module boundaries to see what blocks and what doesn't.
I don't disagree about the convenience of having the Go runtime handle this automatically for you. I do wish core.async could as well.
That said, all your prior comments seem misleading, at least they confused me because it sounded like you were implying something that didn't feel accurate. Core.async has lightweight processes. The core.async Go macro can yield to other Go blocks and it will be really cheap to do so. You can have hundreds of thousands of them concurrently parked and cooperatively switching between each other, etc.
In effect, it means that you can achieve the same scale of concurrency in Clojure as you can in Go.
Yes, it will be trickier in Clojure, because you have to manage the async IO on your own, and you have to delegate an OS thread on your own when blocking IO is needed. In ClojureScript it won't be as much an issue since all IO is async, but still not as nice as auto-yielding by the runtime under the hood as Go does.
core.async doesn't have lightweight processes, for any sane definition of these terms. A code transformation spat out by a macro, subject to all the serious drawbacks I've already mentioned, is not a lightweight process.
But looking at the number of downvotes I've received, stating the obvious no less, I think I'll stop here. I will however take the completely nonsensical, misleading and downright erroneous responses illustrated in this thread as signaling me to steer well clear of Clojure and its community.
Let's not forget that this thread started with someone claiming core.async is better than Go's goroutines.
> But given the number of downvotes I've received here
If I were to guess, I'd say it's because your comments feel like they come from a place of bad faith.
I'll give you the benefit of the doubt here.
> core.async doesn't have lightweight processes, for any sane definition of these terms
We can argue semantics if you want, but that leads to nothing constructive.
I understand very well that the underlying implementation is different. Core.async is a very clever code rewriting machinery. Go actually has a user level thread implementation, with each Go block its own stack, and the Go runtime schedules those over real OS threads. I know all that.
The implementation doesn't matter for what I'm saying though. At the end of the day, it means you can pause and resume multiple logical code blocks and cooperatively alternate between which one is given another chunk of CPU time. All while keeping memory footprint low, and with very fast context switches.
Thus you can achieve a similar scale of concurrency, and even though they achieve this with a different implementation, Go and Clojure both achieve the same end result.
While I totally acknowledge as well that the implementation Go uses allows for a more ergonomic programmer experience.
Here is a very simple example that I think does a great job of illustrating how Clojure was designed to deal with "data" in a straight-forward and concise way:
This is a function that, when given a contents of a CSV file (list of sequences), converts it to a list of maps, where the keys are the column headers.
I'm sure there are clever ways of accomplishing this in other languages, but the default way a Java/C# dev would probably approach this is to create a class that represents the CSV columns, and imperatively iterate over the contents.
With Clojure, the above function is idiomatic, and 4 lines long.
When it comes to "information processing" systems, which is what a lot of us work on, having a language with a primary purpose to provide powerful and concise tools to process that information is.. I don't know, liberating? (maybe not the best choice in words).
(this example was a bit of an eye-opener for me, coming from a .Net background, as I was writing some simple Clojure code that needed to read in some data and was struck at how simple this solution is)
Do you have any recommendations of online resources or books that can get a web programmer (experience in PHP / C# ) up to speed with clojure ecosystem ?
Once you understand the spirit and rationale for Clojure it becomes apparent why other communities don't have this kind of tech yet
Once it gets down to practical things I recommend using clj-Kondo with type hints, Cursive for Intellji, make sure you learn how to hot code inject new code into your running program using your editor shortcuts, and TDD in Clojure is also excellent and immediate: https://cursive-ide.com/userguide/testing.html
Also look out for GraalVM and Babashka we're using it to compile fast native binaries out of Clojure
Its a pretty fun read, and does a good job of covering the language (plus its free, which is probably why most people start there).
You could also take a look at Programming Clojure (written by some of the people behind the language, Alex Miller and Stuart Halloway), which I think is a better resource, but it does cost money.
It can be tempting to want to start in the deep end and try to create something like a web app from scratch as your first attempt at using the language, but there isn't really a Django or Rails for Clojure and you can easily get lost in the weeds (as I did).
My approach to learning Clojure was to start with a simple setup (for me its VS Code with the Calva extension, and deps.edn for managing the project files) and forcing myself to use Clojure for any small utility scripts that I might have otherwise done in bash or JavaScript.
This allowed me to get a feel for the language and how to work with the REPL without having to also digest a lot of information on how a specific library or framework works.
Here's a more or less equivalent implementation in Common Lisp, using alist-hash-table from alexandria:
(defun csv-to-maps (csv)
(loop for row in (rest csv) collecting
(alist-hash-table
(mapcar #'cons (first csv) row) :test #'equal)))
For CSVs with only a few columns (maybe less than 10?), I'd probably go with alists instead of a hash table to save memory, and it'd look like this, using curry from Alexandria:
If you read Rich's paper, there's a section where it says:
> maps were built behind several abstractions. Thus, maps can start with a representation optimized for small maps and switch to a representation optimized for large collections if and when they ‘grow’ beyond a certain size.
Which is pretty cool, because basically the Clojure version of your code abstracts over alist-hash-table and alist, and the most optimal type of concrete map will automatically be created based on the size of the map you are asking to create.
There's actually an implementation of the same HAMT data structure that Clojure uses available in QuickLisp and on GitHub: https://github.com/danshapero/cl-hamt/
Technically, since the hash-table implementation isn't specified in the CL standard an implementation could use a HAMT for the built-in hash tables, but I don't think any do.
The construction of a Map is abstract in Clojure. The concrete type of Map you get is not something you specify. This isn't related to the HAMT per say.
So what I mean is Clojure will use an array backed map when you ask it to construct a small map, and it will give you a HAMT backed map when the map is large.
This is true as well as you add elements to a map, Clojure might automatically promote the map from an array map to a HAMT as its size grows.
"The default way a Java/C# dev would probably approach this is to create a class"
No, not for a long time. The idiomatic C# way would be to use LINQ which actually predates Clojure a tiny bit. Java (since version 8 in 2014) could use Streams to similar effects. Either way the solution would be as short as your example.
Assuming that you are mapping the results to a Dictionary, sure it would be similar.
My point was more that C#/Java developers (myself included at one point in time) would typically reach for classes to store the results, rather than just a Dictionary<String, String>.
It’s not that other languages can’t to this, it just that this approach doesn’t always jive with their conventions like it does with Clojure.
That’s great if your dealing exclusively with CSV files and a fixed set of data types, but the point is that this approach in Clojure is universal regardless of where the data is coming from, or going to.
Everything comes in as a map, gets manipulated with the powerful set of core Clojure core functions, and spit back out as a different shaped map.
It’s a very flexible approach to dealing with data, and that is one of the main appeals of Clojure.
Yup, Java in its current incarnation is definitely a better fit for larger companies that can present more standardized workloads. You want to give each team clean and explicit interfaces that they have to work with, and not have to worry about different shaped maps. Flexibility and power just lets junior developers shoot themselves in the foot.
I agree, I wouldn’t suggest Clojure for a large company with lots of junior developers.
I wouldn’t suggest Java either, though. Go would be a better fit for that situation these days.
I look at Clojure and Go as different sides of a coin.
One is a powerful language the let’s a small team of experienced developers get a lot done with a small amount of code.
The other is a simple language that lets a lot of less experienced developers get a lot done with a lot of code.
I want to stress in this comparison though, that I don’t view one as better than the other.
Go was designed for development at scale, where you have lots of developers, all at different levels, all needing to contribute large amounts of code. In this respect, it does a great job at removing most of the rope that people can use to hang themselves. If I needed to pick one language to use at a large enterprise, it would probably be Go.
Clojure gives you a (different) set of limited functionality, but what it gives you is very powerful, and very foreign to most developers, so you can definitely hang yourself trying to force everyone into using it. If I got to choose one language for myself to use, it would be Clojure.
Go reminds me a lot of the old days of Java, that lead it to its current dominance. It used to be explicit and wordy, but there was usually just one way to do things, so a code base would stay very consistent, leading to much better portability even as code constantly changed hands between developers.
For people coming from a java background, I've had good results with other java+syntactic sugar languages as well, like Groovy and Kotlin, that have accelerated the growth of java to try to pick up some of the best parts of other more modern languages.
I would tend to disagree with the idea of "only choose one language for myself". You want to pick the right language for the job, you want to consider the business needs at hand, you want to consider the team that you have, you want to consider the experience that your team has. Even a company that would prefer to use as few languages as possible could easily end up with javascript for the frontend, ruby for the backend, go for infrastructure, python for ML. And at the end of the day, it's just code, it's just a tool to express your own skills as an engineer.
And then as an engineer personally, you need to get exposed to a range of languages, because each language teaches you a different mindset, and brings with it a different community with different priorities. Not referring to you specifically, but I've seen a lot of junior developers get into Clojure and all of a sudden it's the most amazing language because their life as a Java engineer before was horrible. But it's not Clojure specifically, it's the idea of learning new languages that is itself powerful.
A counter point is that most large companies these days operate with lots of small teams, not one big team. And most systems are composed through a service oriented architecture of some sort, not a single giant monolith worked on by all teams.
So in practice, even at a large company, you find yourself with lots of small teams.
And no such small team at a large company would be composed exclusively of junior developers. There's at minimum going to be a team lead of some sort, and most often you'll have a spectrum from senior to junior.
So in reality, Clojure can be quite a good fit even at large companies. In fact, it might be a better fit at bigger companies, because I think small startup might actually be where you could find a small team of only juniors, and a "deliver at all cost no matter the quality" business need (due to not being profitable yet).
I can't help but feel that all arguments defending how this Clojure solution supposedly can't be done as elegantly in other languages, use "idiomatic Clojure" as the judging criteria. Of course the conclusion writes itself.
In my experience, it's not that you can't do these things in some other languages, it's that you won't, and if you try too, there will be more ceremony and friction.
That said, this is where a lot of people who took a detour through Clojure, come out of it saying when they went back to their prior language or some other language, they felt like they had suddenly become better programmers, and it helped them even in other languages to write better programs. Clojure showed them the way, and now they can recognize it and think of doing it as such in other languages as well.
Maybe a different way to think of it is that you can do OOP in Clojure if you really want, but you won't. Similarly, in Java, you could try and do FP with immutable collections and value semantics, but most likely you're not going too.
And I'm using Java here because in theory, Java and Clojure have the exact same set of available features, they run on the same JVM, same runtime, compile to the same bytecode, can share libraries, etc. So it goes to show that the language itself does influence the style that is most convenient for writing your programs in.
I am familiar with C#, and in good faith spent a bit of time trying to find examples to support this, but in the end using a class is both common and typical (omitting the many StackOverflow results with the same pattern): https://dotnetcoretutorials.com/2018/08/04/csv-parsing-in-ne...
His example is just something like .ToDictionary(column => column.First(), column => column.Skip(1).ToList())* .
The motivation to eventually use a Class would be to make use of static typing, e.g. when using an ORM like Entity Framework (a common usecase: reading a CSV file into a relational database). Static typing is usually considered a strength these days, but it does have its downsides (that debate has been done to death). Showing the parallel Clojure code for that could be very interesting.
However, reading CSVs doesn't require enforcing types, even if one wishes to use a library to deal with the complex quoting cases. CsvHelper is tilted in that direction of mapping to a static type, but it's not the only Csv reading library (I'm a pretty happy user of ExcelDataReader[0] for other reasons, and it doesn't enforce mapping. I'm sure there are more functional-style solutions out there). The original post picked the one place C# uses a powerful functional-like syntax, its generalized query language....
* From my limited understanding, not being a user of Clojure yet (I've been considering learning it for some time, just need to find time to do it seriously), each 'sequence' in the original example is a column, not a row. Were each sequence a row, in C# we could have used the positional lambda argument to get the columns.
I'm a little late to the party, but I don't think I would hire an experienced C# developer that didn't immediately know this could be a one liner:
var csv = ImmutableList<ImmutableList<string>>.Empty;
var map = csv.ToImmutableDictionary(y => y.First(), y => y.Skip(1).ToImmutableList());
I would hope they would then tell me that this will throw an exception when there is a repeat header in the csv data. I'm not sure the functional and safe version is that much cleaner than a for loop:
var map = csv
.Where(x => !string.IsNullOrWhiteSpace(x.FirstOrDefault()))
.GroupBy(x => x.First())
.ToImmutableDictionary(x => x.Key, x => x.First().Skip(1).ToImmutableList());
Edit: But I would hire a developer that knew how to do the equivalent in Clojure without hesitation.
Yes, as I said in my original post (and others in this thread), you can absolutely come up with elegant solutions to this in other languages.
In my experience though, for every one elegant solution like yours, there will be ten developers writing nested for loops.
My example was never meant to be “look at what Clojure can do! You should all be jealous! “. Only an example of how Clojure encourages a different approach to solving problems that we are all familiar with.
Its because the default abstractions provided as part of the language shape the way developers using that language approach problems. In Clojure, the sequence abstraction has been given a lot of thought and effort, which means that’s what Clojure developers will try to use. It also Clojure comes with a rich and powerful set if sequence functions which makes using sequences a breeze and lets you do super powerful things out if the box.
You could simply implement the exact same functions in other languages too and end up with similar code, butsince its not built in, most people won’t think in those abstractions.
Aaaand this works on arrays, but would you know how to make it work for files?
You would need to do something like how the essay wrapped the `rdr` into a sequence, but it would still require (probably) the whole data to be present in the memory at once...
Another language which works differently is Rebol:
csv-to-maps: func [csv-data] [
map-each line next csv-data [map zipmap first csv-data line]
]
Whats interesting about above is that `first csv-data` will be re-evaluating on each `map-each` sequence. Of course you can assign to variable beforehand. Alternatively because code-is-data and vice-versa you can use COMPOSE it inline:
csv-to-maps: func [csv-data] [
map-each line next csv-data compose [map zipmap (first csv-data) line]
]
:)
NB. Rebol doesn't come with ZIPMAP so here's a definition for above to work:
You could definitely do something like this in other languages, probably even in Java/C# using a HashMap or Dictionary, but the difference is that this is “the way” you would do this in Clojure. It’s not, generally speaking, they way you’d do it in the other languages you mentioned.
I think it’s one of the reasons why people always say that learning Clojure makes them better developers even when they’re not using Clojure. The ideas that that take away from idiomatic Clojure and use in other languages.
I think Reagent would be the easiest and most visible example.
Reagent was my intro to React, I had no experience prior nor did I care to learn it at the time. The reagent wrapper around react made it so intuitive to use that I was writing productive (albeit warty code because I had 0 experience with functional programming/lisp/clojure) my first day. I had loads of experience writing Javascript, but at that time React looked cryptic to me without reading any documentation and Reagent did not.
Another commenter mentioned lodash. If I was forced to use JS directly, lodash would be the first thing I reach out for to solve problems.
But in Clojure you also get "cljc" which is a file extension that lets you use code both in Clojure and Clojurescript land simultaenously. With this you can write things like form validation code that's exactly the same front and backend.
The are aspects of solution shapes that Clojure enables that are unique and mostly completely different than other languages, but it is hard to appreciate them if you don't know it. One's intellectual appreciation is literally limited to the languages one speaks (both human- verbal and eg visual, like art, and computer programming).
I look at the benefits more concretely. It is usually possible to express a solution in Clojure with a fraction of the keystrokes another language requires, and Clojure also gives you knobs to control how many keystrokes you devote to the solution vs the stuff under the solution.
You can be all solution domain and be really lean, or you can make some reusable infrastructure to make the solution-side shorter, at the cost of something bigger overall.
The secret of good writing is usually distilled as- tell less, say more. Take away, cut, remove, until you get to the essence.
So many languages REQUIRE you to tell more, with ceremony and patterns and boilerplate. Clojure has some too, but for any given semantic intention, it generally requires the least ceremony.
It is breathtaking to wield a tool that lets you say, in a couple hundred lines, what it takes many thousands in other languages.
It is also hard, and such tools are sharp and can be humbling. But when you arrive at a place where there is no more to take away- I have not felt that "wow" in another language in quite the same way.
I've got a completely contrived example for you, but those are things that are typically easier to do in Clojure than in the languages I know otherwise:
Let's say you're writing a web-application that consumes JSON form an external source. You have to send out responses that accept and reject requests and you add some meta-data.
You have to meet a bunch consistency guarantees in your business logic like: a person cannot buy more than X products at discount Y in a time-period Z. Or some other arbitrary rules.
The external source decides to add an additional field in their JSON tomorrow. With Clojure this isn't considered a breaking change, you don't change a single line of code since you don't care about their new field at all.
Next week, the external source has a bug, they send out messages that are well-formed, but inconsistent with your business logic. You already wrote function specs for those and your messages automatically "explain" why your spec deems those inconsistent. Again, you don't change a single line.
A month afterwards the maintainers of the external source went through a major "refactoring". Now they changed the structure of their messages towards you and you are forced to be compatible. In Clojure this is a matter of adding a new spec and moving around some expressions, or better: composing a new (v2) top level function to meet the new spec.
Again, this is hyperbole, but there is some truth in it as well.
None of this sounds like a problem in a statically-typed system with proper data modelling. In fact for the complete refactor of the external source we would do something very similar to Clojure, i.e. update our decoders for the external data. In fact I'd argue it would be even easier because the compiler would help check all the data is accounted for.
I agree with you in the sense that these things “done proper” would be robust. It is meant as a pointer to what is specifically one strength of Clojure in comparison.
From my experience marshaling/deserialization and bubbling up errors as an introspectable useful message are things that are more involved in statically typed languages than in Clojure.
Typically statically typed languages are very good at internal consistency. But require more ceremony and maintenance regarding outside sources. This is a viable tradeoff in some cases but not so in others.
It all comes down to proper data modelling, whether in a statically-typed or a dynamically-typed codebase. If you were consuming a JSON API that gave you this data:
{"id": 1, "name": "Bob", "age": 55}
But your Clojure app only needed the 'id' and 'name', why would you write a parser function that also grabbed 'age'? E.g. (I don't know specific Clojure libs, so this is for illustration only):
Right, this makes a good point and is worth studying more closely. We have to distinguish these concepts. But from my experience this isn’t done in practice. And by that I mean libraries, frameworks, idiomatic use, culture and so on, which the post at least partially acknowledges (or hints).
This is also a reminder that type systems are not all equal at all. The range of expression matters especially in these kind of discussions.
I feel the other answers to your comment failed to really illustrate the property.
The history paper Rich Hickey wrote alludes to this throughout it all.
Basically, data is central, everything about the language makes modeling data simple, first class, and non-ceremonious. You can take the data your domain relies on, and use it directly, in the same structure the domain structures it, no abstraction needed, no mappings, no adapters, just straight up.
It's really hard to communicate what a data centric approach feels like. But I will try. Keep in mind, there are a multitude of details and features across the entire language which all focus on this data centric approach, and they all serve a role and come together to create this property. It is not any one feature, but really the sum of all of them which enable the property to surface. I will mention only a few to give an idea.
#1 Flexible data representation
Imagine we have a business, and they have product listings, and their products are uniquely identified by vendor and name. In pseudo-model we would have:
vendor+name -> product
Now in Clojure we can model this as:
{[vendor, name] product}
That's it. This will now be the data-structure we will use. We're now going to write a bunch of operations over it which map to the operations the business does in its day to day with regards to its product listing.
Think about how you'd model that in other languages. In JavaScript, which also has a pretty flexible data model, this won't work, because keys to JS Objects cannot be a composite. One would need to have a string encoding and some mapping from the input to the string encoded variant and back, such as:
{"vendor_name": product}
In Java, one would refrain from using a Map<List, Product>, and instead would model this as classes. Maybe you'd have:
public class Listing {
String vendor;
String name;
Product product;
}
List<Listing> listings;
But now you can't easily lookup for a product by vendor+name. So maybe one would instead do:
public class ProductId {
String vendor;
String name;
}
Map<ProductId, Product> listings;
Except it turns out that for certain products, but not all, the business also distinguishes them by manufactured year. And some other are distinguished in addition not by manufactured year, but color. In Clojure you'd just have:
public class ProductId {
String vendor;
String name;
String color;
String manufactured-year;
}
Map<ProductId, Product> listings;
Where color or manufactured-year can be null.
And maybe you don't find the Java one that bad quite yet. So let's now talk about a second data centric feature:
#2 Value semantics
It turns out, in Java, the above code does not work. If the business says, find me the product Kraft MacNCheese, you might be tempted to do:
ProductId productId = new ProductId("Kraft", "MacNCheese");
listings.get(productId);
But the productId you created is not equal to the productId of equal value which is currently stored in your Map. Two objects in Java are equal if they are the same object, not if they have the same value. In Clojure:
(get listings ["Kraft", "MacNCheese"])
That's it. In Clojure, things of the same logical value are equal by default. As such, Clojure has value semantics. Or in other words, equal data is equal, as it should be in my opinion.
#3 Data is same in memory and out
So we've come to the point where we want to export our listings. In Clojure we'd do:
(spit "/home/user/listings/my-listings" listings)
That's it. And when the user wants to import listings:
Because all data in Clojure can be serialized and deserialized as is by default. This gives us back our exact listings data-structure that we had.
#n
And like I said, there's many more such little features all thought of from a data centric perspective. When you add them all up, you start to realize everything is just data needed to be manipulated from one shape to another, and moved from one place to another.
In Java you can (and should) implement equality semantics however you want for Objects by implementing equals/hashCode. If you just want value semantics you can use a record type, which works just like the Clojure example.
That's kind of like saying you can write immutable data structures and then bolt on STM and try to force yourself to stay in functional territory as much as possible in any language, which the paper addresses. You can do this, and Rich did it in C#, but in my experience it's not long before you start to have clashes of philosophy with other libraries you depend on and with the language itself, not to mention people you work with.
Believe me, I tried on one project to write as Clojure-y in Ruby as I could, and as soon as you reach for a library you need to interact with that doesn't play by the rules, things get weird if you want to keep things functional and immutable all the way down. You need to flagrantly violate established idioms in the core of the language, and often you wind up reimplementing things in the core of the language. People who aren't familiar with why you wrote it a particular way will come in at some point later and imperative or OOP it up.
Yes, heavily. Clojure influenced(mostly because for Rich Hickey talks) many parts of the JS functional programming ecosystem. Many functions in lodash and other libraries(immutablejs etc...) are somewhat directly or indirectly influenced from ideas Clojure help make more mainstream. Even Java the language itself has been heavily influenced by Clojure.
So this ecosystems implement the best ideas from niche influential languages, you can use Clojure's "programming model" without leaving the productivity and practical benefits of a big ecosystem. This has been more prominent in JS since the language lends itself better to FP.
This reads to me as a rather superficial take on Clojure.
It's true that JavaScript - or any language, for that matter - can implement libraries that may have been inspired by a Clojure library. But specific libraries aren't what's being talked about here.
The characteristic of Clojure in question - which, IMO, JavaScript does not answer well - is the ability to easily create libraries and DSLs that feel like an extension of the language itself.
To poorly paraphrase a quote that I can't quite remember: Lisp isn't the best language for any problem in particular. What Lisp is the best language for is being a platform on top of which you can implement your own language, that is itself the best one for the problem you're trying to solve right now.
Or, to take a concrete example: In JavaScript, you get optional static typing from TypeScript, which is actually a whole new language that is transpiled to JavaScript. In Clojure, you get optional static typing from Typed Clojure, which is just a library.
The first X was in quotes, so it was a purely lexical reference to the words themselves, without referring to any conventional meaning that those words might have. The second X was unquoted and so by that I meant to convey the usual meaning in English.
Another way of saying it would have been: erm right but experience of Javascript is not experience of Clojure, regardless of any influence Clojure may have had on modern Javascript.
OK. But you didn't ask him to describe his Clojure experience, you just asked whether he was speaking from Clojure experience -- and he said that yes, he was. Asked and answered?
No, on the contrary, I understood his/her reply to be saying that he/she had not used Clojure, but that its influence on the Javascript ecosystem was such that a user of modern Javascript could consider themselves to have "experience of Clojure" in a sense.
We don't have to carry on debating it, but have another read of the reply in question -- I think you'll agree that it was in fact saying that they did not have any experience of Clojure; only of Clojure-influenced Javascript.
I’m in the opposite boat. I went from Clojure to Java.
I got really tired of “finishing” my work, only to discover that it was riddled with bugs from all kinds of weird edge cases. My favorite is having to use (map vec <thing>) all over the place, because the seq abstraction leaks like a sieve.
To be fair, I also worked in a domain that was extremely poorly suited to Clojure. The “data first” approach of Clojure falls to pieces the moment you have a bunch of data that’s syntactically similar, but semantically different. This requires a lot of cross-method coordination, creating a huge mess and a risk of missed cases.
This sounds fishy to me too...
Have you actually consulted anyone, with more Clojure experience, how to avoid the need for `(map vec <thing>)` all over the place?
I would guess you were working with deeply nested data structures and you had to do deep updates, while strictly maintaining the vector data-type.
That's indeed kinda sucky to do just using vanilla `clojure.core`, indeed, but then there is https://github.com/redplanetlabs/specter for that.
Alternatively, instead of dealing with deep nesting, you should probably work with namespaced keys, which allow flattening your data and suddenly a lot of transformations become radically simpler and such a system is more extensible too...
Everyone's problems, context, and experience is different, but from where I stand it sounds like you may have had an a better experience using a data spec library like Metosin's malli.
One of the pain points I had before was different kinds of data flowing through my system. Clojure spec never made complete sense to me and was a bit difficult to write specs that matched my data. It was easier to write data that matched the specs.
Malli does this a lot more easily with human errors that have made my life enormously easier. It's been a game changer for me.
I don’t know Malli; and I haven’t used Clojure since 2017, so things might have changed. So if I’m misunderstanding the purpose of that library, my mistake.
The issue wasn’t that we would use the wrong data in the wrong place. The issue is that we had a bunch of data that was syntactically similar (huge overlap in keys), but semantically different (key numerical results must be calculated differently based on the minor differences between maps).
So the issue is less “the right data got to the wrong place” and more “in this data pipeline there are multiple places where the same data must be treated differently based on minor differences”. This is actually pretty tricky to do in a maintainable way in Clojure; technically possible but really hard to avoid breaking when you need to extend it.
In the end it actually turned out to be an issue that classes and objects solved brilliantly. The solution was to have the pipeline accept an interface (syntactic similarity) and have different classes for the behavior (semantic differences).
This reminds me of a discussion between Alan Kay and Rich Hickey on role of "data" that occurred right here on HN[1]. The TLDR (though the thread is very worth reading, IMO):
Alan Kay argues that bare "data" isn't very meaningful on its own because it leaves the interpretation too open. His view is nicely summarized by the following quote:
> For important negotiations we don't send telegrams, we send ambassadors.
Rich Hickey argues that bare "data" is a fundamental primitive that, while leaving open the possibility of misinterpretation, leaves open the possibility of re-interpretation in multiple contexts. This is the strongest quote I took away from his POV:
> None of the inputs to our sensory systems are accompanied by explanations of their meaning.
Based on your above post, it looks like you're strongly in Alan Kay's camp, which may also explain (in part) your experience struggling with Clojure's approach.
That is a very succinct summary, yes. I’ll add that certain applications are more amenable to one kind of use or another.
I find the possibility of re-interpretation utterly bewildering. I have never re-interpreted my data in Clojure application, at least not in any way that I understand that word. Rewrite algorithms, yes, but you can do that in any language.
The only time I think of re-interpreting data in a different context is in a data lake or similar, which is a wildly different thing, IMHO.
The fascinating thing to me about Clojure is that it's so vocally a "practical" language, and yet is a) a Lisp, and b) purely functional[1]. Usually those two things are associated with language enthusiasts, and not """pragmatic""" development. In the end it works, clearly, it's just very interesting to see that juxtaposition.
I think it says something interesting about the practical value of higher-brow programming concepts, but also about how important the packaging/marketing/ecosystem is to conveying that value and making it accessible to the masses.
[1] I know it's technically not 100% functional, but all of its messaging highlights that philosophy as a focal point and advantage.
I think the most important part about Clojure is how ridiculously straight forward it is.
Once your REPL is set up and you got used to the predominantly functional API and its vocabulary, you are dealing with an extremely simple, data-driven environment that (almost) never gets in the way.
Part of that is due to deviations from traditional Lisps: a more human readable API (first, rest etc.) and less homoiconicity (data structure syntax). But also things like the first class support for names, a baked in, dead simple STM...
For me, this is the reason why it is so practical. I'm not good at top-down programming. I can think of and design a system or solution in an abstract way to get a sense of direction.
But when it comes to writing code I have to look at small, simple things in isolation, look at the results, change the shapes and build something up.
Clojure is very, very good at enabling this kind of dumb, simple, learning-by-doing approach of programming, because everything just works, is easily decomposed and merged together again, with very little ceremony or friction.
And when the simple approach reaches its limitations, then there are these extremely expressive tools like multiple dispatch, macros, transducers, spec etc. which all go beyond what most mainstream languages give you.
This is a minor nitpick but please check your facts about other Lisps when comparing them with Clojure.
> Part of that is due to deviations from traditional Lisps: a more human readable API (first, rest etc.)
first, second, rest ... are defined in Common Lisp
> and less homoiconicity (data structure syntax).
Clojure is no less homoiconic, you just write vector literals in your code; you could write vector literal to represent code in Common Lisp too, but in fact [], {} etc. are not used by default in CL because they are reserved for the user, so that it can be used for domain-specific purposes.
Thank you for those corrections. My limited Lisp experience before Clojure was with Scheme a very long time ago. I shouldn’t have made these false, blanket statements. Those can be harmful for discussion.
In fact I know from reading about CL that it is extremely malleable and that you would be hard pressed to find features that are not present or possible there.
Can someone explain to me why we need a murky word like "homoiconicity" and then explain it as "data structure syntax" (which doesn't clarify much) when "self-modifying code" and "metaprogramming" are crystal clear?
Homoiconicity means you're code is defined in terms of the same data structures that are used in programs written in the language. This makes self-modifying code and metaprogramming easier because the language's standard library is already full of functions that operate on those data structures.
However, you can have self-modifying code and metaprogramming without homoiconicity, and you can have homoiconicity without support for self-modifying code and metaprogramming.
Well, it depends on your exact definition of "self-modifying code" and "metaprogramming", given that it's not like there's an authoratative source you can look up the definitions in. The way I understand the term without additional context, XLST is not "self-modifying code" in the same way malware, certain JIT implementation techniques etc are.
Clojure isn't actually a purely functional language. It's more of a consenting adults language that guides you towards doing the right thing by default. If you write idiomatic Clojure code, then it will be written in a purely functional style. However, Clojure won't stop you from writing imperative code or adding side effects if it makes sense in your particular scenario.
Lisp family of languages also have a lot of tangible and very practical benefits. The syntax is simple very regular, so there's less mental overhead involved in reading it. This helps avoid errors because there's less ambiguity about what the code might mean.
The syntax also makes relationships in code explicit, making it more scannable. In most languages it's not immediately obvious if the lines next to each other are related. On the other hand, s-expression syntax effectively provides you with a free diagram where you can see relationships visually.
The syntax also facilitates structural editing. Instead of working with lines of text, you work with expressions, and you think in terms of moving blocks of logic around. You can see some examples here https://calva.io/paredit/
The syntax also makes it very easy to serialize code, since code is expressed as plain data structures. This also facilitates powerful metaprogramming facilities. You can apply the language to transforming any code written in the language.
Another practical aspect of Lisps is live coding workflow where you connect the editor to the running instance of the application. This creates a very tight feedback loop where you can try things and see the results immediately. This is strictly superior to the traditional workflow where you have to restart the application and build up state to see the results. It's also much faster than using TDD. You can use the REPL for exploration, and then write tests once the code is in the state where it's doing what you want. This is a good read about the workflow https://vvvvalvalval.github.io/posts/what-makes-a-good-repl....
Here's a talk I co-presented last year that discusses an application my team works on and the specific benefits we found from using Clojure https://www.youtube.com/watch?v=IekPZpfbdaI
"... consenting adults that guides you towards doing the right thing by default..." really hits the nail on the head. I do a lot of Java interop at work, which is a necessary evil, but once I'm in Clojure code I'm happy to be gently pushed to be more functional, immutable, have less state, etc etc.
Also, thanks for all your great work Yogthos, Luminus is an amazing contribution to the Clojure ecosystem as well as all your other libraries!
If you read the history, you'll see that small scripts was never a use case targeted by Rich Hickey. It was focused on information systems programming, where a few seconds of startup time is a blip on the full running time of the application.
That said, slow startup times are only true of Clojure JVM. ClojureScript and Clojure babashka, for example, both start fast enough for small scripts.
Also, seems the JVM is slowly improving in startup time department. I was surprised to see that a hello world is now down to 800ms start time on my 2011 laptop with Java 14. It was 3s before with Java 8 on the same laptop.
For comparison, ClojureScript hello world was 550ms, and babashka hello world was 36ms. Python hello world was 68ms and Ruby was 111ms.
It's true that Java hello world will be faster than Clojure JVM hello world. That said, a big Java application with lots of classes to load, and lots of static initializers (which would approach the same number as what Clojure requires to be loaded and initialized) would take as much time as it does for Clojure.
So when people say that part of the slow startup is Clojure's fault, they mean more that it is implemented in a way that the JVM isn't very fast at starting. But if you look at Clojure implemented on other runtimes, like ClojureScript, the startup times are much faster, because the NodeJS runtime is much faster to start and initialize everything than the JVM is.
In effect, if the JVM improves startup times, so will Clojure JVM's startup times be improved. That's why I see this drastic difference between a Clojure hello world on Java 8 vs Java 14.
Startup time was also a deterrent for me initially, but that was ~7-8years. I annoyed the heck out of me.
I was doing Rails back then and that was too much too.
You can further reduce this overhead if you are just using the the Socket REPL, but I think the best support for that (which allows evaluation interruption for example) is the Chlorine plugin for the Atom editor. I usually just accept the overhead of nREPL though.
Startup time for Clojure itself has become pretty fast. My guess is you were using something like lein or boot which involves more than just starting up Clojure.
Either way, for scripting uses there are tools like babashka or lumo with sub second startup times.
Great story, really enjoyed reading. Rick Hickey is a great speaker (if you haven't watched his talks on youtube, do it! They are totally worth it), and certainly a great programmer, which is why I respect his opinions on dynamic typing, even if I stubbornly disagree (one day I might still dive into Clojure, but I am diving into the arguably opposite end of languages right now: Rust!!). I have even more respect for him now knowing that he actually worked on Clojure for years without any income!! This is a work of passion, and I admire him for that, and for being able to turn that passion into a company big enough to pay his bills and that of several of Clojure's early adopters.
FWIW, I don't think it really does to treat his opinions on typing as a blanket, universal, prescriptive statement that one can or should agree or disagree with according to one's own blanket, universal, prescriptive position. He generally doesn't come out and say this explicitly, but, in his talks, he's always speaking to a certain audience, about a certain class of problems.
You couldn't come up with a better pair of languages than Clojure and Rust for illustrating this sort of thing. I like them both (as far as my extremely limited experience in each will allow, anyway). But, if a language like Clojure is a candidate for the thing I'm working on, then I would never have dreamed of picking a language like Rust. And vice versa.
To add to that and concur that Rich generally doesn't have such un-nuanced strong opinions without context, see his talk Effective Programs. He explains what Clojure is good for and isn't good for, and says more clearly it so depends on what you're doing.
He doesn't organize the Clojure conferences, but I'd imagine that if he didn't see any value in any context for static typing, there wouldn't have been a talk about Typed Racket and static typing researchers as keynote speakers, etc.
And FWIW, spec is a library trying to come as close as possible to the imaginary midpoint of static and dynamic typing, but in a Clojure-y way.
>I have even more respect for him now knowing that he actually worked on Clojure for years without any income!!
Hickey mentioned in one of his talks that he actually burned through retirement savings (at least some?) while taking these years to do Clojure. I got the impression he had been thinking about it for a while prior and that it was in the neighborhood of two years. (The context was, he was thanking his wife for helping to make Clojure possible, in part by indulging in many converstions about very minute details of how Clojure would work.)
(I have no idea if this is covered in the paper as it's 46 pages, I've bookmarked it but not yet read it.)
In those 46 pages, he goes into details regarding how the idea came up to him, why he took a sabbatical of 2 years (which ended up lasting to this day, he says at the end :D ), what he did during those 2 years (not only Clojure) and how he basically has no retirement savings left because of that.
There was also a group of people who wanted to see Clojure succeed who donated money ($100 each, if I recall correctly). But I would guess that was a drop in the bucket compared to what he had to spend of his own money.
I remember being at the first Clojure/conj - I had just spent some time learning a bit of Common Lisp and stumbled onto the conference announcement somehow.
This was very fortunate for me - after attending the conj, I was able to use Clojure at work to sneak in some FP on the readily-available JVM platforms. I later used it to do some internal REST-API work. The code for that project has run for years without modification or error.
For whatever reason, I also discovered that thinking functionally with Clojure worked so much better and naturally for me than object-oriented design methods. While I have drifted away from Clojure over the last couple of years, I find that my problem solving mind is much better in other languages because of the time spent thinking functionally with Clojure.
"The code for that project has run for years without modification or error."
To follow up on this point, if you ever need ammunition to sell the use of Clojure inside an enterprise level organization, just show your Chief Architect the graphs of code stability over time for Clojure (pg. 71:26) and ClojureScript (pg. 71:30).
After working with JavaScript (not trying to bash) for the last few years, the idea of stability has become top of mind for me.
Looking at it, seems like the Scala code base found some stability in 2011?
This could be a factor:
> On 12 May 2011, Odersky and collaborators launched Typesafe Inc. (later renamed Lightbend Inc.), a company to provide commercial support, training, and services for Scala. [0]
I was at a talk by Guy Steele where he mentioned in passing that Clojure was a Lisp which had done everything right. This was all the more impressive to him because Rich Hickey had until then been a relative outsider to the Lisp/Scheme community.
I think what Rich got right is the practical approach. Clojure is about compromises (like all engineering). It's not a theoretical exercise, it doesn't prove a point, it's about getting things done.
This also means that sometimes things do not seem as "beautiful" as one might like. Well, that's because they aren't. There is no way to sugarcoat certain things without making the language less practical, and Clojure seems to have been designed with no sugarcoating in mind.
Incidentally, I love this quote from Rich, an answer to people complaining that Clojure isn't easy to learn: "Instruments are made for people who can play them". I've grown to appreciate that over the years. You don't expect to be able to sit at a piano of a particular brand and just start playing it.
I just wish it didn't target the JVM. I think if it had a native runtime that loaded quickly like Python it would be more popular. I love Clojure and really wish I could use it, but I just don't do the kind of things that can justify running a JVM.
This is definitely a valid criticism! Two ways that this may be getting better: GraalVM's native-image and babashka.
babashka is a little interpreter that implements large portions of "scripting Clojure" so you can write tiny Clojure scripts the way you'd write bash or Python.
GraalVM's native image is a Serious Compiler (unfortunately also with Serious RAM appetite and compile times, though this has gotten much better over the last few months) that will produce native binaries on Windows, macOS and Linux.
Planck uses the os native webview for execution. Lumo runs on node. Eden is compatible with most clojure libraries, and is meant for embedding. Babashka is a clojure interpreter written in clojure and compiled with GraalVM. I've used Planck and babashka for shell scripting and prefer babashka.
He explains how he created Clojure explicitly as a hosted language. After the JVM implementation, it quickly got implemented for CLR, and then JS (ClojureScript), and both are full implementations, not just look-alike.
By all accounts, though, Clojure became as popular as it is because of the JVM, not despite it.
Btw, Rich had two versions of Clojure - Java (JVM) and .NET (CLR) and shortly before releasing it he decided to focus on the Java one, and deleted the .NET one.
ClojureCLR still exists and is somewhat maintained (last commit was last year, but then again, little has happened in the clojure world in the past year so I'd still use it): https://clojure.org/about/clojureclr
I started out with that view too. Now, after many years of using Clojure, I have come to love the JVM: so many of the libraries it comes with have become critical pieces of my daily workflow.
I find this very interesting, because it seems the programming landscape has changed quite a bit since Rich Hickey's world where he conceived Clojure.
How he describes information systems, in that world, JVM is an amazing piece of technology that can hardly be surpassed.
But today, the programming landscape goes far beyond that. ML and Data Science, infrastructure automation, serverless, etc.
I also work in the world of enterprise information systems, so I'm always surprised about people saying the JVM doesn't work well for them. For me the JVM is the best piece of tech out there, only matched by the CLR. Even Go isn't up to par yet.
But I reckon, other use cases value different execution characteristics. Low memory footprint is more important than raw performance. Throughput is more important than latency. Binary size is more important than portability. Startup times is more important than runtimes. Etc.
Well the good news is that Clojure is meant to be hosted. And the JVM is only one of its many hosts. ClojureScript can run on NodeJS, eventually Deno, it can run on JavaScriptCore and in the browser. All these offer alternative execution models which can meet a new set of requirements.
Clojure CLR is still real and robust, while it hasn't seen much use, it is ready at any time to take over. If the CLR provides to fulfill alternative execution characteristics it would be an option. Right now, the CLR has pretty much the same characteristics as the JVM though, which is why its use in Clojure hasn't seen a lot of popularity.
Babashka and Joker are Clojure based runtimes, interpreters really, so exactly the same as what Python and Ruby offer. Babashka especially is picking up pace, and already starting to offer a really compelling set of alternate execution characteristics, specifically one where startup time trumps all.
I'm also quite fascinated by Clojerl, a Erlang hosted Clojure. While less official, the maintainer is very active, and I feel it's actually now ready to be used in production.
And there are more, smaller ones, but I'll skip those.
What this shows is the landscape of possible runtimes is vast and will just keep expanding. Right now lots of experiment in having a Clojure hosted on Go, on Common Lisp, on Rust, etc.
This a testament to Rich's hosted design. Clojure is easy to build on top of other runtimes. This ease really helps it grow and reach more use case.
And there's the fact that the JVM is only getting better. GraalVM already offers native binaries with a small memory footprint, small binary, fast startup and still pretty fast performance. There are so many new garbage collectors coming along. Some for large heaps, small heaps, no heaps, tuned to throughout, latency, low memory, large memory, etc.
P.S.: Also, I really urge you to reconsider a JVM. The truth is, it's such a powerful piece of technology that it can be hard to navigate and understand even what all it can do. I believe a lot of people are turned off by that and choose to use Go or Python because they have only one easy mode of operation. But the JVM can do so much, it's a worthy investment, it's very possible it has what you actually need.
For example, you no longer need to install a JVM. Apps are self-contained. The user doesn't have to install a JVM, and the bundle contains only what the app uses, not the whole JVM. And as mentioned, compiled native binaries are also a possibility now.
You mentioned ML, so I just had to link to the awesome libraries by Dragan Djuric like ClojureCUDA, ClojureCL, Neanderthal, Fluokitten, Bayadera and recently Deep Diamond
JVM is not to blame here, it's Clojure that takes a while to load and initialize. Java actually crushes Python in every aspect in terms of performance (which is to be expected of statically typed language).
GP specifically mentioned startup time. It's true that most of the startup time for Clojure projects is a Clojure-specific interplay of things the JVM does (but also not language-intrinsic: ClojureScript on V8 does not share that problem, for example), but default flags CPython still beats default flags OpenJDK for "start and do nothing".
it's not the static typing that makes a difference, it's the compilation. As an extreme example, asm is very much not statically typed (hell it's not even considered typed in most circles), and it beats the pants off of Java.
As another, less flippant, example, Julia will typically beat Java in most long-running algorithms (but probably not time-to-first-x), because it doesn't have a well-optimized-for-compilation-speed compiler, but it does have a better-optimized-for-running-speed compiler for many scientific tasks. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Last year I spent about twenty minutes writing mine (Julia #3) in the n-body example and it tied the naive java. I'm not even a Julia programmer anymore.
I can't, sorry. This is my paraphrased recollection of his response to an audience question from years ago, and a quick search doesn't turn up any recordings.
Yea Clojure is like the Python of lisp style languages. Good language design and it can interact with Existing Java libraries. (Python does the same thing but with C).
It requires jvm. But you don't have to interact with java, sometimes its easier to just use a particular java library, but many people report never having to touch java. JVM seems like a nice to have, a lot of the standard java tooling works for clojure as well (profilers and what not).
Nit: CPython, not Cython. Cython is a whole different thing that leverages CPython to compile Python programs to C, as well as compile Cython-the-language, effectively annotated Python, to faster C.
CPython comes with its own VM: are you saying the issue is "you need to install 2 things instead of 1"?
I think, though, that the things that Clojure gets right go way beyond just the JVM, though. Clojure's approach to nil punning, for example, is really well thought out.
"nil punning" has been a feature in Lisp since the earliest days, but as far as I know it's only called "nil punning" in the Clojure community.
I don't have anything against Clojure (well, I'm a little negative on the JVM), but I personally don't see any reason to move to it from Common Lisp.
On the other hand, I'd miss CLOS, and I feel like CL has a better/more consistent design. Something about Clojure always feels like they're "winging it".
In a nutshell, nil punning is a thing in lisps (and some other languages) where the null value can have different meanings in different contexts. It's comparable to Python's concept of "truthiness" and "falsiness", but more so.
A great quote from the introduction, on why Rich took a sabbatical to work on Clojure:
...to give myself the opportunity to work on whatever I found interesting, without regard to outcome, commercial viability or the opinions of others. One might say these are prerequisites for working on Lisps or functional languages.
The results are also astounding, because what he came back with produced extreme commercial viability, and is excellent in the opinions of others.
Having used Clojure professionally for a couple years now, I feel so blessed. It is really well cut out for a whole host of problems that are common but extremely labour-intensive in conventional languages.
Maybe one of these days I'll get time for my own sabbatical, and solve the two major tasks I see with Clojure: taking the performance from good to excellent; and hosting it in a robust systems language suitable for implementing well-trodden functions.
Yeah, share both common concerns, centering around the JVM (which I love, and have used and studied since it launched, but it is baggage).
There have been a number of attempts to build Clojure on top of Go (I worked on one), and another on top of Rust. Both lack the dynamic runtime ergonomics that Clojure leverages. Clojure is also seeing significant performance benefits, both runtime and startup, from the Graalvm, but that also will have limitations.
I am wondering about D, which should be sufficiently low level from a systems perspective but may also have sufficient dynamic capabilities.
Believe it or not, this is relevant to an ongoing sabbatical project of my own. I'm grossly unfamiliar with Java, so it's often hard to read through the Clojure implementation :/
I worked on one privately back around 2014, then later picked up on some of this work: https://github.com/venantius/glojure, done by a friend of a friend. Neither one got anywhere useful. I more became convinced that to get to a useful place it would be necessary to generate (and compile) code and although Go has some of that tooling, I did not have the appetite to build with it.
The Java implementation implementation is hard to read even for Java folks, it is more of how a C++ programmer would write Java, both in terms of syntax, and semantics.
I'd love to use a light-weight Clojure hosted on Go, etc., but also wonder whether performance improvements to the JVM will solve many of these issues without requiring a full rewrite.
Many of the issues Clojure has with the JVM are shared by regular Java apps, so I think there is motivation to improve many of these areas over time, even if it takes a bit.
I think the problem with Go is that it is invested in some oddities that are of a similar scale to Java oddities. For example, from my ignorant perspective, Go's multiple return pattern for error handling seems to have turned out in the long run to be a much bigger mess than Rust's Result.
AFAIK Rust ditched the optional GC long ago, but I think a lot of the infrastructure that has enabled async/await, in the way of hooking in a “runtime” through attributes and macros, could help with building a GC into a rust host. I think this could be a special strength if you combined the futures infrastructure with the allocation logic, maybe with a local spill/scratch pseudo-heap for allocations that don't escape.
Think ztellman's aleph and manifold interfaces, but based on tokio, with some of the allocation magic (or deallocation magic, depending on how you slice it) of BEAM.
I think I've been a bit let down by some “Clojure-inspired” languages which do not have HAMT or CHAMP datastructures, or do not have good implementations of them.
The JVM's profusion of high-quality garbage collectors is another thing that is hard to compete with; though I think there are massive opportunities with a collector similar to Shenandoah 2.0, if the allocator can be aware of the properties of Clojure.
Furthermore, a purpose-built runtime would be an amazing way to explore transparent non-volatile memory. I think there is staggering value in transparent access to datastructures that are significantly larger than memory, especially if that access is inside a future/task run by a NVM-aware executor, and can wait safely. Think MapDB, but persistent, immutable, indistinguishable from the standard Clojure datastructures, and integrated with the garbage collector.
Of course, as I've not written anything significant in Rust, I will shut up about it beyond this, because there's nothing more obnoxious than people extolling the virtues of something they've never accomplished anything significant with.
Heh, I actually like the "comma-ok" pattern and use it in Clojure.
IMO one of the areas of abstraction leak in Clojure is around error handling. Most of the time the options- out of band and out of code path, like Exceptions, and in band, like null- don't sit well with me. I usually want in-band, explicit, with the capacity for some sugar so if necessary the error cases can be organized in one place. Comma-ok does this for me without any magic, and I would like to see it as a lowercase-p protocol integrated into things like the threading macros (there has been some work in this area...)
But high level, comma-ok is just data, not a type- another area of abstraction leak.
I hear the interest in arriving at a more efficient GC but don't have much current background in that area. I did a lot of GC work at the configuration level with the JVM, and read a bunch of the source a decade or more ago, but recognize that's not my area. I similarly spent a bunch of time in the linux kernel world 15 years ago, and still check in on the mailing list from time to time, but that's a different life.
Agree with non-volatile mem opportunity, though note that was the intent with virtual memory architectures and then CPUs went and ruined it with the cache hierarchy. From a performance perspective if your code is hot, cache-aware will noticeably outperform non-cache aware. Figuring out ways to make this automatically efficient at runtime in the context of also supporting GC would be a fascinating PhD project.
> When I first was learning Common Lisp I was appalled at the tolerance on Usenet comp.lang.lisp of arrogance, hostility, intimidation and other anti-social behavior of the presumed ‘smartest guys in the room’. I was determined (a) that the Clojure community would not be hosted there and (b) that kind of behavior would not be tolerated.
Thank you, Rich Hickey!
The persistent data structures and functional core library are great, but many would not bother with Clojure if they had to deal with the toxicity that is present in the Common Lisp community.
Comp.lang.lisp was awful, and the trend somewhat continues even though it has gotten better. Looking through newb threads on various lisp forums there are a few people that always seem to try their best to be mean or funny on someone else's expense.
It is not that hard: if someone writes awful code, point them in the right direction. Don't claim they have malfunctioning brains.
I stopped writing CL a few years ago even though I enjoyed it. For my personal stuff scheme is even more fun, and the communities are a lot more pleasant.
I've been working with Clojure for a few years now, and the thing I love most about the language is how it nudges developers to write straightforward code. It's maybe closer to a well-structured procedural style than to highly indirect OO or to deeply polymorphic category-theory-inspired typed functional programming. The emphasis on mostly-first-order pure functions composed of other functions, receiving and returning immutable values, makes it easy to untangle even the worst balls of mud, while the well designed set of data-structures (few) and functions operating on them (many) allow for easy and terse expression of everyday data manipulation chores.
If we keep the majority of the code pure and straightforward, we can deal with state and I/O with other constructs, like atoms and records, only where we need to (doing this systematically tends to lead me to some variation of the functional-core/imperative-shell pattern).
STM, multimethods, macros and whatnot are cool and have their uses, but after a while I think the really cool thing is how little do we need to resort to fancy stuff and how much is accomplished with just functions and data.
Lol I’ve been learning Clojure for fun these last few days and I had to read this twice because the first time I thought I was looking at a function from a module.
> we can deal with state and I/O with other constructs, like atoms and records
Can you elaborate on what you mean here or link to specific articles? I’m talking about the representing I/O side effects as records part.
There is a component framework that builds dependency tree for stateful modules and does implicit injection of params. It is a beautified pattern(usually you can do initializations in higher levels, but you have to resolve dependiences yourself and for the whole tree) for dealing with things that require state while keeping explicit references(so that you understand where it came from and can inspect(visually) its lifecycle).
I was super excited to see this paper released. I've spent time with Python, Java, C#, Javascript and limited time on C and Rust. Clojure was the largest paradigm shifter by a large margin.
Having done concurrent programming in Java, and even implemented some concurrency primitives like semaphores, Clojure was incredible to use. For those that don't know, Clojure uses immutable data structures that mitigate concurrency issues. If you are curious how that's even practical, I highly recommend the paper and also reading about Persistent Data Structures. They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.
On top of that, you up reducing lines of code by 60-70% of traditional Java or C#.
> They reuse shared data, similar to how Git only tracks the delta's between commits, rather than multiple, full copies of the file.
I learned about immutable data structures at Rich's conference talk years ago, and the whole concept blew my mind as well. However, what you said above is not strictly correct. The idea is not to store deltas (which is what Subversion does), but immutable copies of objects (in case of Git identified by their SHA-1) that can be used to build multiple versions of a hierarchical data structure (e.g. a list, tree or hash map). Git commit is basically a tree of other objects with some metadata. The problem with deltas is that they need to be iteratively applied to some base snapshot which requires computation (and probably a cache as well), while immutable objects are easily accessible in constant time.
Thanks to Clojure (and ClojureScript), I was able to build my self-funded SaaS business. There is no way I would have been able to tackle the complexity without Clojure.
I also love the mature approach and mature community. It is, quite simply, a different world from most other software places, and one I enjoy a lot, as I'm mostly focused on building and shipping a large and complex codebase with an extremely small team.
Can you elaborate on how Clojure helped you tackle the complexity? I've considered learning Clojure (I have the Brave Clojure book sitting not far from me) but I have concerns about the learning curve (I'm a fullstack dev by day (C#/Angular) slowing me down in my own attempt to build a SaaS.
I have a similar background and in Oct 2018, I started spending lots of time learning with that exact book! I first learned Emacs (covered in the book) and it really wasn’t bad at all. I did some tiny fun projects in Clojure, and some exercises on 4clojure. After a couple of months I understood the language and could easily make sense of most of what I saw except core.logic which hardly comes up in practice. Even Heroku has first class support of Clojure, so deployment is trivial for MVPs.
You likely won’t have an issue learning the language, but from my experience, making full stack apps in Clojure is a bit more challenging (compared to .NET and Flask). It’s challenging to figure out what libraries to use for your app, since frameworks are hardly complete and usually outdated in Clojure.
I see nothing but praise for Clojure and for good reason. I experimented with it a bit a few years ago and it seemed to be a more practical and useful version of Lisp, although I say that as someone who wasn't that experienced in Lisp more generally, with only some small experiments in Scheme. Something just as significant was that the ecosystem seemed to be quite sane and I got on really well with Leiningen for example, if you were considering diving in then give it a go.
I personally won't consider it now but not because of the language or the ecosystem but because of the runtime(s). For me the layers of abstraction are too much complexity to not be justified.
Also a lack of documentation was always a problem, especially around clojure script.
I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...
I kind of know where you are coming from. 'Back in the day', early Clojure adopters would ask one to read SICP before asking questions. SICP is indeed the best CS book I have read but that attitude was uncalled for.
"'Back in the day', early Clojure adopters would ask one to read SICP before asking questions."
I was in the Clojure usenet from very early on and do not recall this happening. That's not to say that it never happened but it was certainly not a common response to questions. Instead, I remember early adopters like Chris Houser, Chas Emerick, Phil Hagelberg, and many others taking a lot of time to help people who had questions with the language.
I've also received countless hours of help from @alandipert, @micha and @jumblerg on the Clojurians slack about boot, Hoplon and Hoplon UI.
Without them I couldn't have successfully built a 6 ppl Clojure team, who was consistently delivering working software!
> When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...
Clojure is not meant to be hard to learn and you are definitely smart enough. There are many, many learning resources for Clojure in every form and it has one of the most helpful communities I'm aware of on Clojurians slack, etc.
> I also don't like smug community. When you say about the lack of documentation and get told that's a good thing, clojure is meant to be hard to learn, it's not for everyone and maybe you just aren't smart enough...
Where did you experience this? That is unacceptable in most of the Clojure communities I know, and the only big one I don't know personally is the subreddit.
What tooling has given you problems? I think that some tooling assumes familiarity with the Java ecosystem (which is a problem for many things in Clojure land), but otherwise it seems pretty solid to me.
I have posted about this before but Clojure does have tooling problems specifically ergonomics.
Look at create-react-app. Two commands and a ton of editor integration across intellij, vscode, vim, etc gives you incredible access to a rich ecosystem and plugs together really nicely with sensible defaults.
Clojure has some answers in this space but everything feels bolted on and not nearly as polished. A big part of the problem is the size of the community, there just isn't enough hands to build out the infrastructure that other langs enjoy.
I still love Clojure and will continue to use it but in order to do so I have to understand that some simpler things will just be more work.
That might be because you've left out CIDER, which is
1) the most popular Clojure programming environment
2) solidly documented
3) built on top of Emacs, which is in itself very well documented (Just the reference manual of the vanilla Emacs is 500 nicely written, if dry, pages)
4) very featureful.
I guess the biggest problem is that most people are not familiar with Emacs, and not willing to read any documentation and guides.
I've never understood why so many people are reluctant to learn Emacs (or Vim), but then don't see any problem with learning a new language specific IDE every few years.
While we can suggest Cider as an option, I have had nontrivial problems with breakage on updating, Emacs falling over with long lines of text, Windows support, etc. I also know that I am not the only one who has had these problems https://twitter.com/ztellman/status/1055152840589496320.
These kinds of issues a part of the tooling gap and are real reasons people give up on Clojure and I do find the dismissive attitude of most when I have suggested this in the past off putting.
Addressing Emacs specifically, while it would be probably worth the investment it is not an answer most people want to hear and when we suggest that Clojure is this great thing it is important to show people on some of their terms because learning a new editor on top of Clojure more then we should be asking IMHO and for all the talk of Clojure being pragmatic this feels directly opposed to that ethos.
Let me emphasize, I don't want to say anyone is doing anything wrong or that Emacs isn't the right way forward but it had to be acknowledged that there are real problems that path that need improvement when it comes to onboarding new developers and with daily use.
In 6 years that I've been using Cider daily, I don't remember it being broken after update. Ditto for Emacs.
A few times Cider had major changes, but even then it was quickly resolved, and was relevant only to people that were using daily snapshots. I guess if someone is a beginner, they'd use a stable version, and these are rather stable.
Big files could be problematic in Emacs. OTOH, Clojure files are never big.
As for Emacs requiring a bit of learning at first. Well, yes. It has lots of features that beginners have probably never encountered in other tools. These features are awesome. They also help a lot. So, someone expects to unlock a great featureset, but refuses to spend even a few afternoons learning about that featureset. I understand that people would rather get something out of nothing, but it's not Emacs's fault.
Part of the problem could have been that I was using Spacemacs and package support and that may have broken it but circling back around this connects with meeting people halfway, I don't want to learn Emacs style control, I know Vim, Vim works elsewhere, and I just want to write Clojure. I can do that now with Cursive so the trade off isn't worth it. A lot of people learning feel and think this way in my experience, especially if I selling Clojure to someone who is reluctant, anecdotally at least 4 people I know have followed this path, I am sure it extends beyond them.
Per the big files, its not so much Clojure but output at the REPL. Especially as a noob accidentally grabbing way to much data for a buffer is really easy, Emacs hangs, REPL and process dies and has to be rebooted and you have to bring up everything again. This is non-trivial friction when learning and should really be handled gracefully by the editor.
As for everything else regarding Emacs and learning all these points stand and are valid but still don't address the problem of someone wanting to learn Clojure and instead having to learn Emacs + Clojure + Cider. For someone just starting on the path most won't have more than 3 files of CLJ and may not even interact with Lein or Deps. Refactoring IS important but not to them. I just don't think the trade off is worth it for someone just toying around with CLJ in their free time. It may not be the best metric but "time to hello world" and "time to blog" matter to people coming in because it shows that the promise that they are sold is something that they can reach and achieve.
Hm, I use Spacemacs particularly because of the easy config.
That said, you're right: you should just be able to use vim, and historically CIDER has been the dominant (most polished) environment. At Clojurists Together Foundation, we're trying to fix that though: a lot of our funding has gone towards IDE development including for vim. Some of it towards CIDER also benefits vim because they share internals.
It's worth mentioning that BBatsov, the creator of CIDER, maintains a practically zero-config Emacs distribution, Prelude. A novice interested in Clojure could install Prelude, and start coding Clojure and other popular languages using pre-configured packages in, let's say, 15-30 minutes. The difference from Spacemacs is that Prelude is much simpler, does not mess with Emacs defaults, is close to vanilla Emacs, and has been super stable for years.
aaand I guess that's what ppl mean by "ergonomics issues".
Emacs is very stable, quite logical in some sense, but also hasn't kept up with the times and doesn't default to those typical keyboard shortcuts, which most newer editors almost all share...
but this has been discussed countless times before on twitter, clojureverse.org etc...
My bet would be that you are working with the Linux version of Emacs.
That's indeed a lot less problematic and more performant too.
Under macOS and Windows, the situation is not that rosy and since ppl want familiarity, they typically mix in something like Spacemacs, Doom Emacs, etc into the mixture. For a long time they would probably won't even hear about Prelude or if they do, they would not even assume that's a preset system for Emacs. I've seen this happening to a couple of ppl already just in the past 2-3 years.
Mostly that there is an interplay between the config that gets spit out by CRA and the editors that consume it, I don't need to give intellij the run config for the 400th time, it's just there by default. I click run and it runs.
An exercise for you to try that might illustrate it. Setup create-react-app and get hello world up and running. Now do the same with Clojurescript. That should include a way to hot reload and edit because that is often the next steps.
Notice the friction that CLJS still has and the aimlessness of starting from a build script as is suggested on the Clojure site vs getting a functional build in two commands.
I don't want to come across as ungrateful for the hard work that David and Alex and the whole team put in, they are the only reason have dev tools that feel comfortable to me and I can't thank them enough for that, but I just want to point out that friction and difficulty and to try and get people who do understand to put themselves in the shoes of developers who don't and to acknowledge that we as a community can aspire to more.
I am not familiar with CRE, so I could be missing something, but based on my understanding of what you write, `lein new re-frame hello-world` should do the trick. Do `lein dev` from the project root directory to get the app running with hot reload.
Just tried this and the install size is 796Mb of which 513Mb is node modules. The initialised git repo is a whopping 274Mb - WTF? I re-initialised git, excluding node_modules and it's now 244k. I recommend the author leaves `git init` to the user.
Is your criticism that it downloads to many packages, or just that they're (I agree, incorrectly) being added to the git repo? Because if you want to create your own minimal setup, shadow-cljs' tutorial will help you do that: but OPs criticism was that there's no easy starting point that has all the bells and whistles, so yeah, this has a lot of bells and whistles :-)
That is extremely fair but again it doesn't have deep hooks and the convenience that Javascript and CRA have with, for example Intellij. This isn't a problem that is easily solved, Colin is only one person and JetBrains is a huge team, but it is still a gap that Clojure needs to have a story around to attract more developers and to make our lives as devs starting new projects easier.
Can you help me understand the deep hooks that need to happen? My understanding is that it's stuff like, intellij's "run" knows how to call `npm start`. Elsewhere in the thread you highlighted specific requirements like getting hello world running and hot code reloading with your editing, but CCA seems identical to CRA in that regard.
There isn't enough officially supported first class tooling. Most things that are a given for Rust/JavaScript/Go are community maintained with only one maintainer in Clojure, development is slow and features take a long time to roll out. (Let's not delude ourselves that Webpack level of donations is even possible here) This is a problem with every language that is small but it is particularly problematic with Clojure as it has a commercial company behind it but you get the feeling that they are disinterested in promoting the language and ecosystem beyond selling their database software. Go also espouses "frameworks bad, libraries good" but the biggest difference is that they actually have a strong robust ecosystem of solid libraries while pretty much every Lisp claims to have production-ready libraries but expect you to cobble together the entire kitchen sink from scratch.
> development is slow and features take a long time to roll out
As Rich has said, Clojure is small and intends to remain so. Clojure has everything you need right now. It doesn't need gobs of new features to be sufficient. Most Clojurists are hard pressed to come up with things they really need in the language itself, so the rate of change in the core language has slowed. (And the rate of change in the tooling and libraries around Clojure gets more attention now.)
> they are disinterested in promoting the language and ecosystem beyond selling their database software
... is incorrect. We have every desire to promote Clojure as a language (regardless of Datomic). Could more be done, sure that's always the case. Please help promote Clojure by doing great stuff and telling people about it!
I also believe that tools.deps is a step in the wrong direction, fragmenting an already small community and forcing all tools to support yet another dependency/build system.
Leiningen is super powerful, and it was never made clear what kind of conflicting requirements / philosophy led to the birth of tools.deps etc. In absence of an official answer, the presumed answer is Not Invented Here Syndrome.
That documentation does not explain what prompted the creation of another tool when leiningen was already (and still is to a large extent) the de-facto deps/build tool.
> Go also espouses "frameworks bad, libraries good" but the biggest difference is that they actually have a strong robust ecosystem of solid libraries while pretty much every Lisp claims to have production-ready libraries but expect you to cobble together the entire kitchen sink from scratch.
I mean, Java is pretty much the biggest ecosystem there is and it's trivial to use Java libs from Clojure, so I'm not sure this is a fair comparison.
The Go ecosystem seems pretty weak outside of the standard library. I'm not an expert, but I was trying to fix something the other week and was shocked by how few packages there were to solve my problem. The main one (which is a dep of things like k8s) is currently unmaintained.
Clojurians slack is honestly one of the most welcoming and understanding programming communities I've had the pleasure of joining. I frequently ask stupid questions and will occasionally have library maintainers take their time to fill my gaps in understanding. The tooling complaint is valid, but it has been getting better every year. There are plenty of good books for Clojure too, like Brave & True that help you understand the documentation.
I used Clojure a lot back in the beginning (contributed a little money, used Clojure fairly intensely on two long term consulting projects). Before that I used Rich's Common Lisp to Java bridge project. I think his impact goes far beyond just Clojure users, having helped push immutable by default, etc. as standard good practices.
I moved away from Clojure because I wanted to move away from the JVM. I did a fun project, a nutrition/cooking web app [1] about 12 years ago in Clojure, but I am slowly porting that from Clojure to Common Lisp for another example program for my book [2]. Perhaps not as well known, but Clojure joins other less commonly used languages like Haskell as languages with fairly good support for deep learning. I feel like any language that does not have a good deep learning story will be limited. Also interesting that Swift has such good access to Apple's deep learning libraries and also a version of TensorFlow. I encourage Clojure developers who work on deep learning support!
You can subscribe to Dragan Djuric's deep learning book [1].
He is using the subscriptions to fund work on Deep Diamond [2], a fast Deep Learning framework with GPU support for Clojure.
I've been here for the entire journey, but no one can summarize it quite like Rich. And I bet he could write twice that just on nurturing a community - He spend hours and hours on IRC in the early days, coaching, teaching, correcting and even admonishing bad behavior. He's truly done something remarkable in both building this powerful language that I use daily, and a very helpful, friendly and intelligent community around it.
A great read! What I really appreciate, beside all the technical stuff, is the effort to create a positive and helpful community around Clojure.
From the paper: "When I first was learning Common Lisp I was appalled at the tolerance on Usenet comp.lang.lispof arrogance, hostility, intimidation and other anti-social behavior of the presumed ‘smartest guys in the room’. I was determined (a) that the Clojure community would not be hosted there and (b) that kind of behavior would not be tolerated. Early on, on IRC and the mailing list, I established and required a tone of supportiveness, positivity and respect."
I fell in love with Clojure and fell out of it. It is a great language. Lisps have this inherent elegance to them especially in data transformation pipelines.
That said I found its performance still a bit on the wanting side particularly if you avoid type hints. For many tasks the performance is probably acceptable but the backend work I tend to do makes the abstract style get in the way of trying to figure out performance issues.
Also I'm coming around to the imperative style more and more. Sometimes a for loop is all that's needed and makes for a more understandable code than all those hipster maps and folds.
I am curious because I am doing real-time financial data processing with sub-millisecond latencies just fine in Clojure. Admittedly, there’s no heavy number crunching in my use case.
I work with service to service integrations. Data gets ingested into Kafka, transformed and delivered to target systems. The initial prototype used Clojure but a plain Java reimplementation ended up much faster - 2-4x difference.
I also found that Clojure is full of weird edge behavior. Sets are unsafe, and might change the type on you. Seq leaks like a sieve, so there are all kinds of places where you have to map seq, and type hinting is the worst. The moment you type hint a Protocol, all kinds of weird shit starts to happen, including the need to import the type hinted classes in the consumer (except Java.util.*), even if you don’t reference it. Oh, and if you type hint protocols wrong, it doesn’t error, but your performance remains bad.
At least it did in 2017. I haven’t used it since then.
I don't know what "sets" refers to here - idiomatic Clojure does no such thing (there are some facilities with the word "set" in them but they are all rarely used). seq is not leaky at all imo, it's a model that choose to use (https://clojure.org/guides/faq#seqs_vs_colls). Or you can choose not to use it now with transducers and use any of the seq functions and collect into a specific collection type. I don't know what you're talking about with protocols, typically you don't need to type hint them, and I think there may have been an issue with unqualified imports on referred functions but that was fixed several releases ago.
Set as in the collection. Sets are idiomatic, they even have syntax sugar for their construction: #{}.
Seq is extremely leaky, as whether or not a built in function takes a seq or a specific collection is undocumented and arbitrary. Hashmaps for example do not accept seqs when using “into”, it accepts vectors. So my code was littered with (into {} (map vec <thing>)). That’s a leaky abstraction.
You need to hint protocols if you want to go fast.
I have trouble understanding almost all of these complaints. I have been developing Clojure for two years now at two different shops and haven't encountered the issues you describe.
I have to concur that it's not easy to gather what's idiomatic and why.
It took me watching the "transducers", "inside transducers" and "reducers" talks several times, before i realized that `into`, `transduce`, `eduction`, etc are the same family of functions, providing transducing contexts.
Despite that, I have no idea what are those issues in the critique above.
To me it smells like trying to do something forcefully, instead of stepping back and approaching the problem differently, in simpler ways.
Other way to avoid such issues is to formulate the problem clearly and ask it on a couple of forums, where more experienced Clojurists can answer it.
This of course takes some time, sometimes hours, but it yields great results. I've received very insightful responses directly from authors of libraries on Github issues or Clojurians Slack and from Cognitect too about some advanced Datomic query technique.
I've noticed that some ppl, instead of asking for help, sweat out some convoluted, inconvenient way to solve a problem, then decide it's the only way to solve it in a specific language or with a library in question and they just keep copying that solution and teaching it to others too and constantly complain about it...
It feels to me that OP might be in a similar situation to some extent.
I worked in Clojure for 2 years in a company that had people who’d used Clojure for 4-5. I spoke at the Conj before I got a full time job in Clojure. Of course we researched and experimented, and I find the suggestion that we didn’t do our due diligence extremely patronizing.
I’m more than happy to explain the issues I had, but don’t imply that the problem is that I don’t know any better and I’m doing it wrong.
I can recommend YourKit. It's a paid tool but if you really need it, the small cost is totally worth it.
Usually the bulk of your perf. problem comes from a few hot places that are called often and the JVM needs to do reflection on those every time. Type-hint those and the performance usually goes through the roof.
A lot of issues centred around persistent immutable data structures that Clojure uses for just about everything. Yes those data structures are faster than any previous attempts at language level immutability constructs but are still much slower than their mutable counterparts. Yes, you can use in Clojure Java Collections through the interop interface but that mostly defeats the point of Clojure.
In the end I retreated back to Java but even that is wrought with a number of performance issues due to its lack of contiguous arrays of complex types. I’m toying with Go as it’s a simple language that has far fewer performance surprises than most other popular high level languages. If that turns out disappointing I’m going back to where I started 24 years ago which was C++.
Or you can also use the transient/volatile versions of data structures and functions, but I wouldn't know where to learn about these in depth. I've seen them being mentioned briefly in some official docs on clojure.org and seen them in implementations of basic functions, like `reduce` and whatnot, but I'm still not very confident using them... Also, I had no need for more performance than what I get with the immutable data structures.
Section 2.4 of the paper gave me a chuckle. Like Rich, I've yet to write a program that didn't involve some kind of database. Organizations care about data not programs. Writing programs just gives them a way of applying organization-specific rules, in the way of transformations, to the data.
I've gone ahead and restored the article, but not knowing much about Rich Hickey I hope someone here can edit the article to actually contain some (sourced, i.e. with citations) content, as it's pretty bare right now (looks/looked like this: https://en.wikipedia.org/w/index.php?title=Rich_Hickey&oldid...).
The article was created in 2008 and its deletion was discussed and happened in July 2011: https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio... Someone recreated it (without discussion) in August 2013, and it was reverted to its "redirect" state in January 2014. I suspect that notability has changed now (e.g. Clojure itself is more popular, there are other things, and if nothing else his notability as a speaker is probably worth mentioning) and my guess is that if the article were to be discussed again, it might survive. But this is assuming there is actually content for the article that someone can add to it! Otherwise what's the point of an article about a person that doesn't say anything?
In my experience, adding something well-sourced to Wikipedia is usually a more lasting contribution (probably more people will read it, over time) than comments on HN -- the only downside is that it may get reverted quickly on flimsy grounds and you need some experience to make good edits that will stick.
Edit: Or just mention here what can be added to the article and what the source of each bit of information is, and I'll try to add them!
- ClojureScript (https://clojurescript.org/)
- Datomic database (https://www.datomic.com/)
- edn data format (https://github.com/edn-format/edn) - extensible data notation (subset of Clojure's syntax)
- Transit (https://github.com/cognitect/transit-format) - format for conveying values between languages
- Fressian (https://github.com/Datomic/fressian) - extensible binary data notation, used by Datomic
- REBL (http://rebl.cognitect.com/download.html) - graphical tool for browsing Clojure data
- Codeq (https://github.com/Datomic/codeq) - Clojure+Datomic application to do code-aware queries on git repos
Thanks! I've added most of that (there are no Wikipedia articles for Transit/Fressian/REBL/Codeq, so not sure how to mention those without external links which look weird inline).
Still, an editor who doesn't like the current state could simply revert it, as there are no sources for any of this. Note that the criterion is “verifiability, not truth” — Wikipedia is supposed to be a summary of what has been said about a topic/person in books, newspapers, etc., so even though all of these are obviously true, it would be best to find a source that mentions these things and cite that!
[Similarly, in general, if you are the subject of a Wikipedia article and you find that it has some incorrect information about you, then rather than editing the article directly it is best to get something published or write something yourself—even a blog post—and cite that. :-)]
That's great. Thank you for going ahead with that.
I'll try to add some more relevant details with citations over the weekend or atleast put it here.
Do you think if we start a new thread, it will gain more traction? Just a thought.
> Restoring redirect - I see no evidence that he is notable for anything but Clojure
Redirecting the Wikipedia article for Elizabeth II to buckingham.co.uk. I see no evidence that she is notable for anything but being the British queen.
He's notable as a thinker and speaker about programming, quite apart from Clojure. The idea pool is the same, but that's true of any thinker.
It's a bad argument regardless. Plenty of figures in intellectual history are notable because of one big thing they created, and I'm sure most have Wikipedia pages. Alas I'm blanking on examples just now.
Aren't most notable people notable for one notable thing? Sorry, to get into Wikipedia, you'd have to win at least one gold medal at the Olympic games, be elected president of at least two countries, die in no less than 3 battles, AND you have to be the first confirmed case of at least one pandemic.
I think a rule of thumb is that something must have been written that was about you, rather than about your creation or whatever. The idea is that as Wikipedia is supposed to be a summarization of the secondary sources on a topic, there should be some material that would fit only in an article about you, and not in the article on whatever you're famous for. Some interesting examples are in the section about “people notable for only one event”: https://en.wikipedia.org/wiki/Wikipedia:Notability_(people)#... (which is not about people who created something and does not apply here! Just mentioning it for what it shows about interesting aspects of their policy.)
Definitely. I've come across plenty of Wikpedia profiles that are just self-maintained profiles by hardly known entrepreneurs or startup founders. Ranging from Clojure and Datomic, to his insightful talks on software design, and everything else in between, Rich Hickey's made far more significant contributions to the world in comparison.
What is notable about Hickey besides being the creator of Clojure? Not to say that this alone doesn't make him notable, but I also don't really see the gain in an entry that is just a glorified redirect.
All right... all right... but apart from better sanitation and medicine and education and irrigation and public health and roads and a freshwater system and baths and public order... what have the Romans done for us?
I also think that it's separately notable that Rich Hickey is a prolific speaker. Granted, he almost always talks about Clojure, but I think a lot of people outside of the language still watch those talks, and it would make a lot of sense to have something to look him up by.
Thanks to Rich for designing and thanks to Alex & Rich (and others) for maintaining. It is my favorite language thus far out of (Java, Ruby, Python, Bash, Awk, Erlang, Swift, Tcl and Objective-C).
As a language it is a work of wonder and it focused and refines thinking about programming. Learning it will make you a better programmer in whatever language or paradigm you more commonly use.
As a tool for building things it varies. I only know about the web, and on the web it's probably best as a secondary language to do heavy lifting as a service rather that the backbone of a SaaS, where Django or Rails will get you where you want to be faster. Trying to do fairly common web stuff in Clojure is a lot more work and happens at a much lower level of abstraction, even if you use curated selections of libraries like Luminus. The focus on libraries over frameworks is enticing for some engineers who want to spend a long time building reliable and totalized systems, but for quickly building out features it leaves a lot to be desired. The ecosystem also assumes you are going to make an SPA in ClojureScript. (That said, Fulcro is fascinating if you are committed to the SPA way of doing things; the way it manages data flow seems unique, at least to me.)
IMO, the ClojureScript SPA ecosystem is a stronger competitor in its domain than Clojure on the backend.
Re-frame in particular is a gem. It's as though someone tried the React/Redux stack, thought long and hard about actions and selectors, and realized that with one or two more pieces, everything falls into a beautiful, purely functional harmony.
Clojure on the backend is awesome because you can use mature, stable, pure Clojure libraries for almost everything and it works together beautifully.
ClojureScript on the other hand suffers from constantly having to interop with JS libs and tools, or trying to wrap them, with many wrappers being outdated or having a bus factor of 1.
I feel like the advantages of CLJS have eroded over the past few years:
- TypeScript has fully taken over, generalizing the advantages of static typing
- JS libraries have adopted functional patterns, narrowing the comparative advantage of CLJS
I've used it quite a bit for side projects, but never professionally. I definitely think it's a great general purpose language. However, I don't use it as much anymore because I feel like my primary coding tasks (webapps) are better served by Elixir. As a language I vastly prefer Clojure. But it's hard for me to pass on BEAM and the concurrency model therein.
I have been using it for almost a year now as I use it at work. I like it a lot and am glad I made the switch (I spent the previous 8 years doing C# work). We use both Clojure and ClojureScript
Yes, and yes there are, and many are worth the brain bending to get them absorbed, and the brain bending is different for each.
Speaking personally, having worked at one level or another with, looks like, 48 of the top 50 TIOBE languages over 30+ years (lacking OpenEdge ABL and ABAP) and about a third of the next 50 (where Clojure resides), Clojure is near the top of the list for me in terms of brain-bending value.
I did one personal project in it just to get my feet wet. Unfortunately the tooling was not great at the time so it soured my impression a bit (I'm told it's gotten better since then).
But also, I just have a really hard time giving up static types these days. Even for small projects, I always feel anxious about code that doesn't have annotations on function boundaries. It may have more to do with personal sensibilities than actual need, but regardless, it makes things unpleasant for me.
I have come to appreciate static types for browser programming(UI development) and won't go back to a dynamic language in that area by choice. For server side stuff, like a HTTP API, Clojure really shines IMO. But then you hit the database layer and the benefits of Clojure diminished greatly, compared to just doing the thing in JS. You could extend Clojure's "programming model" to the database with Datomic, but choosing it is not decision to take slightly(for various reasons), it also has many production horror stories.
Clojure has good foundations but the ecosystem just isn't there.
Same here: having gone from C++ to JS+Python and back to Golang/Rust, I just can't imagine life without types.
However, of all the not-static-typed language ecosystems I see, Clojurists seem to miss types the least. I have dabbled with the language and I know about `spec` but I don't quite seem to grasp why even shops using Clojure without spec don't seem to miss types much either.
Perhaps people more experienced with Clojure can shed light on this?
A pure functional style does help somewhat; things are more predictable and consistent, expressions can be evaluated inline to test them out, etc. And I think Clojure teams (Lisp teams in general, really) are already forced to be on the small side because of how flexible things can become with macros and such. You have to have a stronger shared headspace compared to, say, a Java project. So that also lends itself to being more dynamic-types-tolerant. Finally, that very flexibility (with macros, etc.) is only fully possible in the absence of static types. So there is a genuine tradeoff.
Still, even with all of the above I'd rather have them than not have them.
It depends what you want to use types for, really. If your motivation for using types is provable correctness and you can grok Haskell (or better yet, Idris), then you should do that. If you want performant dispatch, Clojure can give you JVM-level perf through the use of protocols. If, however, you want input validation (in the sense that you can be sure that a function argument you've been given will allow you to do what you intend with it) then static typing isn't the only player in that game.
In my experience (which I think resonates with RH and most Clojurists) is that for the vast majority of _information-driven_ systems, types are used as the latter. If you have a `Customer` class that gives you guarantees about the availability of, say, an `accountNumber` field, that is useful for correctness as you can be sure the information you need is there. However, if some future downstream coder wants to use your customer in the more general sense of being a human, then (s)he has to worry about sub/super/abstract-classing, may have to make upstream modifications to expose previously hidden data, and similar faffage. In Clojure the idiomatic solution to this problem is to "just use a map"; the real-world downside to this, however, is extremely weak contracts between functions. In this example, what `spec` allows you to do is strengthen those contracts by verifying that the data your function is provided is sufficient for your uses (as in the map contains all the keys you'll need with suitable data types in the fields) _without_ constraining what downstream consumers can also do with this data (extra map keys are ignored).
These checks are only done at runtime and only when enabled, however writing a spec gives you (very-nearly-almost) free generative testing that will run your function a default of 1000 times with random data in the correct shape to make sure it doesn't blow up—this isn't a guarantee of correctness, but it does provide extremely high levels of confidence (most type systems are also not even close to guaranteeing correctness either, only compliance with the type system). Spec also gives you (for free) performant runtime coercion for use in actual real-world code. You get a lot of bang for your buck.
`spec` is definitely not a type system, but it very capably fulfils a similar role in the kinds of programs Clojure was intended to be good at. It gives you all the flexibility and dynamism of Clojure with most of the confidence of static typing, without constraining either.
> by verifying that the data your function is provided is sufficient for your uses (as in the map contains all the keys you'll need with suitable data types in the fields) _without_ constraining what downstream consumers can also do with this data (extra map keys are ignored)
This assumes a very class-oriented type system. Duck-typing and the like don't have the upstream ontology problems.
My main desire from types is as an iteration assistant (with editor integration). Even if I wrote a function myself, I may not remember the exact order of arguments, or the exact name of that one property on the returned map. I want to a) be able to quickly peek and see what those things are - either by visiting the definition or, even better, via a pop-over in my editor - and b) have my editor tell me immediately if I did something dumb so I can correct it and keep moving.
In a dynamic language, whenever I need to double-check the contract for some code, I can't just go look look at its type signature, I have to go read through it. I have to fully load that whole subtree of information into my brain (recursively to any functions it may itself call), when I'm really trying to focus my thoughts on something else. This can be a huge, needless drain on mental resources.
Spec would help with this some, assuming the author follows a good convention of putting all of their assertions at the top of the function. But maybe those assertions are done inside conditionals, creating a more complex type. And maybe my editor doesn't know what to make of them (do any editors? genuinely curious). Etc. It just creates a bunch of little speedbumps to cognition that add up.
> In a dynamic language, whenever I need to double-check the contract for some code, I can't just go look look at its type signature, I have to go read through it. I have to fully load that whole subtree of information into my brain (recursively to any functions it may itself call), when I'm really trying to focus my thoughts on something else. This can be a huge, needless drain on mental resources.
Yes, this is a headache, and certainly a problem that afflicts Clojure. Spec doesn't really help much in this regard. There is a proper static typing system for Clojure[0] that does provide a lot of the editor integration you speak of, but as I recall it was a little too brittle and orthogonal to Clojure's way of doing things to be as useful as spec. Some of Clojure's core constructs are completely impossible to type statically.
As with everything there are tradeoffs and choices to be made. I've been writing Clojure professionally for 5+ years now and there's no other language I have much interest in dealing with full-time (yet). One has to choose one's poison I suppose.
There's also destructuring which both "extracts" local variables from data structures and serves as an informal documentation/description of the data shape.
You can combine type-hints and destructuring for a very powerful effect. Nowadays, I almost never have a problem "remembering" what or what shape of data I need to pass around.
If you want something to let you know you've made a silly mistake like putting a string in a number function or missed an argument use clj-kondo it will catch those problems for you in the editor
If you want to make a Person class or type I would recommend instead making your information model as a Datomic schema and have all your functions use associative destructuring with defaults values for missing data with function comments for good measure
For dev I would recommend Datahike over Datomic initially
I can't believe people that actually actively used a compile-time type-checked PL in a real-world long term project can even consider using a dynamic language for anything serious.
I've worked with both, and i wouldn't ever move away from the ease of refactoring and confidence that a statically typed language offers you.
Consistency in Clojure comes mainly from things like:
FP is the default idiom, state is very isolated and explicit.
The language is dead simple and often declarative.
The way you write Clojure programs is by attaching a REPL to your editor and by evaluating expressions in isolation. And you'll be running a linter like clj-kondo[1]. This has a similar effect as running a type checker (and it in fact does catch a lot of things that a type checker does).
Clojure spec can express much more than most (mainstream) type systems, is itself introspective and can be used for generative testing.
First class support for names: Keywords are not explicitly attached to their containers/data-structures but can be optionally be attached to namespaces and specs. This means a lot of problems that arise when changing code are simply avoided in Clojure. They still carry around the semantics and consistency you gave them, regardless of where they are used.
I've been working on a side project since about a year now and I can't remember when I last had a type error.
An idle wonder: one non-criticism of Clojure (and lisps in general) is "the parens." Anyone who's used Clojure finds that a) this is totally handled by the editor, and b) the parens are actually an advantage, because all language structures are the same, which means the same hotkeys, etc, can be used universally.
Now the reason I mention this is: I "hate" static types. But then again, the only place I've actually used them was when writing Java in plugin-less Vim in college ten years ago.
Am I the outsider who just got a bad taste because I was lacking proper editor/IDE support?
I'd actually point the finger at Java itself, though I imagine the editor didn't help. Java's type system is (IMO) about as cumbersome and unhelpful as they come. A good type system should be minimally obtrusive while preventing lots of programming errors (both when writing code and, perhaps more importantly, during maintenance by a different person a year down the line).
For languages with good type systems, I'd recommend something like OCaml (or maybe F# - I know it's very similar and probably has a better ecosystem but I haven't personally used it).
Elm also seems decent - it's designed to be pretty simple and has JS interop but it doesn't have good tooling. Typescript is an alternative that has an okay type system but pretty quality tooling.
I'd recommend staying away from Haskell even though it's commonly recommended. The type system is quite complex and its use of monadic I/O makes writing "real" programs, even toy ones, difficult for many people. If you want to learn Haskell, I'd say start with OCaml/F#/Elm. Those are all very similar and will let you learn the fundamentals of ML-style programming before diving into Haskell-specific details.
I would also add that type inference is one of the biggest differentiators when it comes to developer experience. When you have to manually type every single thing, even local variables, it's easy for the system to hinder more than it helps. Whereas in TypeScript you're free to omit return types whenever you want them to be driven by the internal code, or specify them when you want to establish a stronger contract. The same goes for local variables. You can even call generic functions without parameterizing the generic type if it can be inferred from the arguments.
You got a bad taste because Java tastes bad. :) Almost any other statically typed language will be more pleasant than Java. Java barely counts at all, in my biased opinion, because null references are so pervasive. OCaml is a better intro (or if you're into the new hotness, Rust). But yes, tooling helps too.
I would say it's more that your experience is with Java. With Java, proper IDE support is pretty important, but mostly just because it's so verbose and clunky.
If you try something like OCaml, you really don't need any special editor support (though it's still nice if you have it).
Given that so many us do, perhaps you should look deeper? I've worked with both and I find no loss of ease of refactoring or confidence, and a significantly easier code base to reason about.
I've been really wanting to learn another language (been doing TS for the past couple years, would really like to see what else is out there). What do you think are other great languages to learn instead? I'd want something ideally with: functional paradigms (really want to learn these), good tooling, good ecosystem, good production deployment recipes, shared code between front end and back end. Any thoughts?
This is superb, and I'm only on page two. I'm thankful that Rich finds the time to produce his thoughtful content and I hope he has the opportunity to present this at HOPL-IV when it is rescheduled.
The distinction between Information Systems and Artificial Systems is the key point to me and reflects my experience, not that I understood it at the time.
I use Clojure today because it is a better thinking toolkit for Information Systems (or the inelegant 'Products' as I defined them in my conj talk). I'm an order of magnitude more productive in Clojure than I was in Java. It's not a small step, it's a step-change.
All right. That's it. I'm porting over all my code to Clojure.
Any tips to get me started? I'm writing a web app to process DAG-formatted data.
EDIT: Looking at re-frame + reagent 'cause I have some React background. I also found The Joy of Clojure and Elements of Clojure from other threads. Anything else I should know before diving in? (Past experience: Haskell, Python, C#, JS)
I'd say also that the ML-dialects, such as F#, Ocaml, and SML, are pretty close to a statically typed Lisp/Scheme, at least in the flavor of programming aside from them having better pattern matching built-in.
I'd like to see a specifically designed statically typed parenthetical language, basically something that merges Racket and F# but simplifies things a bit.
In the context of the domain I have the most experience with, web applications, I'd summarize the bulk of programming as defining data, data transformations and shoving data through APIs. Once you get the hang of it, Clojure makes it trivial to transform datastructures from A to B. This has enormously expanded the kinds of problems I can solve.
When Rich says "It was impossible to avoid the sinking feeling that I had been “doing it wrong” by using C++/Java/C# my whole career."
I feel somehow that this is the case for the majority of people but they don't realize it yet because their experience is the most popular trendy language or framework. I've seen many examples of libraries in different languages having enormous amounts of commits and hundreds issues for problems that are trivial to solve in Clojure.
I was in the same boat, constantly grabbing for frameworks and if one wasn't available that made my task simple, would struggle trying to bend the frameworks or language I was using to come up with a solution.
I'm not a language geek and I don't write code outside my work for fun. I want to spend the least amount of time possible in front of the computer and sleep knowing my web applications won't topple over. Clojure has fit me so well that I don't think I would have accomplished what I have in other languages.