Since they cite me (and my essay from 2014) as part of their decision making process, I want to throw in 2 cents here. They ended up deciding on Go, whereas I have ended up preferring Clojure, yet I agree with a lot of what they say, so I'll try to clarify why I ended up with a different decision than what they made.
I understand what they mean when they write:
I think the first time I appreciated the positive aspects of having a strong type system was with Scala. Personally, coming from a myriad of PHP silent errors and whimsical behavior, it felt quite empowering to have the confidence that, supported by type-checking and a few well-thought-out tests, my code was doing what it was meant to.
There are times when I appreciate the strict type-checking that happens in Java. I do get what they mean. But there are also a lot of times when I hate strict type-checking (in particular, when dealing with anything outside of the control of my code, such as whimsical, changing 3rd party APIs that I have to consume (for some business reason), or even 1st party APIs that feel like 3rd party APIs because they are developed by another team (within the same company) or for some reason we can not fix the broken aspects of some old API that was developed in-house 6 years ago.) Because of this, I have become a proponent of gradual typing. If I am facing a problem that I have never faced before, I like to start off without any types in my code, and then, as I understand the problem more, I like to add in more contract-enforcement. This is what I attempted to communicate in my essay "How ignorant am I, and how do I formally specify that in my code?" [1]
I think everyone who works with Clojure sometimes misses strict type-checking. Because of this, there have been several efforts to offer interesting hybrid approaches that attempt to offer the best of all worlds. There is Typed Clojure for those who want gradual typing, and there is more recently Spec. Given what I've written, you might think I am a huge fan of Typed Clojure, but I've actually never used it for anything serious. The annotations are a little bit heavy. I might use it in the future, but for now, I am most excited about Spec, which I think introduces some new ideas that are both exciting for Clojure, and which I think will eventually influence other languages as well.
Do watch the video "Agility & Robustness: Clojure spec" by Stuart Halloway. [2]
I also sort of understand what they mean when they write this:
No map, no flatMap, no fold, no generics, no inheritance… Do we miss them?
There are times when we all crave simple code. Many times I have had to re-write someone else's code, and this can be a very painful experience. There are many ways that other programmers (everyone who is not us, and who doesn't do things exactly like we do) can go wrong, from style issues such as bad variable names to deeper coding issues such as overuse of Patterns or using complex algorithms when a simple one would do. I get that.
All the same, I want to be productive. And to be productive in 2017 means relying on other people's code. And, in particular, it means being able to reliably rely on other people's code -- using other people's code should not be a painful experience. Therefore, for me, in 2017, one of the most important issues in programming is composability. How easy is it for me to compose your code with my code? That is a complex issue, but in general, those languages that allow for high levels of meta programming allow for high levels of composability. Both Ruby and Javascript and Clojure do well in this regard, though Ruby and Javascript both have some gotchas that I'd rather avoid. In all 3 languages, I find myself relying on lots of 3rd party libraries. I use mountains of other people's code. Most of the time, this is fairly painless. But there are some occasionally painful situations. With Ruby I run the risk that someone's monkeypatching will sabotage my work in ways so mysterious that it can take me a week to find the problem. And Javascript sometimes has the same problem when 3rd parties add things to prototype, perhaps using a name that I am also using. I so far have had an almost miraculous time using Clojure libraries without facing any problems from them. It's this issue of composability that makes me wary of Go. While I sometimes crave a language that simple, I can't bring myself to give up so much of modern languages best features.
> If I am facing a problem that I have never faced before, I like to start off without any types in my code, and then, as I understand the problem more, I like to add in more contract-enforcement.
This seems to be a widespread sentiment.
In practice, I've found that prototyping anything remotely complex without types is so painful that I'd rather settle for an inferior design than trying to come up with the best possible abstraction.
In Haskell, I come up with a coherent skeleton without having to implement mundane details, hit a wall in the design space because of some case I didn't think of, come up with a better idea and then go back to the code and refactor with confidence. The type system always guarantees that my prototype is coherent as a whole. And I can do that dozens of times.
In Clojure, even with spec, I'd have to implement all my functions fully before being even able to test the design as a whole (sure, testing individual functions works fine in the REPL.) And after hitting a wall, having to reimplement everything every time is just too much work.
> In practice, I've found that prototyping anything remotely complex without types is so painful
Just to offer a counter-point, I have a Clojure project here with 3k LOC, without using spec/schema. All I have is 700 LOC tests. The tests enforce semantic meaning, along with (some) contracts. I miss types from time to time, but it is no where near as bad as you mention. Against me is the fact is that my app is mostly self-contained, and written all by me. I am fairly certain the project would be atleast 30k LOC if I wrote it in Java.
I think value of types only comes into being when there are many people working on a single code base. Repl/tests/integration tests will take one a long way before it reaches its limits.
I don't buy the argument that types are useful for large codebases, because
1. Types don't enforce meaning. Meaning is far complex than type. Haskell style types only work as far as they enforce meaning.
2. Types have limits too. More accurately, humans have limits. Working with large codebases where there are 1000s of types is hard; which it shouldn't be because that was type systems selling point all along.
Cleaner abstractions of code, separated by strongly enforced protocols(types) is the way to go, I think.
Have you tried what I'm talking about, i.e. prototyping something by specifying the types and iterating on the core design first before implementing large chunks of the required functionality?
It's probably hard to see the benefits without having tried it first.
This [0] is one of the bigger Clojure projects I've done (around 2.5kloc), also without spec/schema, and I really didn't dare refactor much, even when having a clear understanding of how things could be done better. It was just too much work.
With types I'd go through 10 design iterations before settling for something I'm satisfied it, and even halfway through the project changing things radically isn't a problem (I've worked on a 50+ kloc Haskell backend service, and changing core data structures used pervasively throughout the codebase was a 10min job, literally.)
I see what you are trying to say. You are saying Clojure is not the right tool to do top-down design. I agree with it. It is however a very good tool to do bottom up design.
See https://www.youtube.com/watch?v=Tb823aqgX_0
Today I caught a bug in a macro-expanding code walker, where it was expanding the wrong form. The syntax being walked is (foo-special-operator x y z . rest). x and z are ordinary forms that need to be expanded; y is a destructuring pattern (irrelevant here). The walker was expanding z in the place of x: that is to say, expanding z twice, and using that as the expansion of both x and of z. That's simply due to a typo
referring to the wrong variable.
The type system would be of donkey all use here, because everything has the correct type, with or without the mistake. The code referred to the wrong thing of exactly the right type.
A lot of code works with numerous variables or object elements that are all of the same type, and can be mixed up without a diagnostic.
Types don't enforce all meaning ie., the enforcement of contracts through types go only as far as they mean something to the problem you are applying it to. It does not cover all the complexities of the problem, or the way the code is changed in the future.
EDIT: This is also why it is easy (and nice) to implement parsers in strictly typed functional languages, because parsers are well studied theoretically. The problems in the real world are not studied well enough for contracts enforced via types to work completely.
Types don't enforce meaning, types are a tool I use to enforce consistency of meaning along certain important dimensions. I actually find that even more important when the situation is messy, because I'm likely to initially mischaracterize some aspects of it initially and when I go to change things it's very useful to be told what's now inconsistent.
I get what you are saying. What I'm trying to say is typing takes too much from me, in terms of complexity over-head, that I'm better without. I found this is true in practice now. As I said before, I write tests to do what you say types do - for me, that is enforcing meaning. Types do allow for easy refactoring, and I think that is weakness for untyped languages.
> The above function enforces that getting a user can fail and you must contact the outside world to get a user.
That seems sort of backwards. It enforces that the caller be able to handle failure (and similar for IO). It may well be that "getting a user" doesn't do either (e.g. `pure (pure defaultUser)`)
> In Haskell, I come up with a coherent skeleton without having to implement mundane details, hit a wall in the design space because of some case I didn't think of, come up with a better idea and then go back to the code and refactor with confidence. The type system always guarantees that my prototype is coherent as a whole. And I can do that dozens of times.
Could you go into a bit of detail about this approach? Do you mock functions out and just specify type definitions, and fill in functionality as you go?
Note that src/Hotep.hs exports a bunch of undefined functions. I've been able to verify that these types all make sense even without implementing a thing. As time goes on I may learn that the implementation drives the types somewhere else and then the compiler will make refactoring easy.
However, already I've gone through about 5 iterations of this design which drove me to debug some structural questions about the whole affair and also dive deep into Erlang documentation to determine how they solved problems. These explorations and their results are encoded into the types.
At this point I'm beginning to consider implementation and I can keep filling out just partial implementations against these types. I'll probably make 2 or 3 toy implementations to test out the ideas again with more strength before moving on the final ones. The whole time the types will be guiding that development and helping it move quickly.
Key to this whole affair is the need to describe types utterly before a completely successful library is made... and also the ability to defer the burden of providing type-checking code for as long as desirable. Haskell supports this wonderfully—even more wonderfully with things like Typed Holes and Deferred Type Errors which enable a really great interactive experience I haven't yet needed to employ.
I've used Scala, Clojure, and Go. I found Scala to be too feature-rich for its own good.
It was fun to write (Look at me! I just spent two hours figuring out how to compress this old Java 7 function into a one-liner in Scala!), but reading someone else's Scala was almost as mind-numbing as reading another programmer's C++.
Go is... meh. Quick to learn, easy to write (and read), but you quickly hit a plateau as far as personal productivity goes. I can see the value when working on large teams, but on my personal projects (for which I have limited time) my own productivity is paramount (not to mention I want a language that's fun to use) :)
Thus I've found Clojure is my ideal language for the time being. It strikes a good balance between power and simplicity (and at this point I find it more readable even than Go ).
The way you describe your experience with Scala makes me think you only had a very superficial look at it.
At it's core Scala is very simple & the syntax is very regular, far more than Go or Java and a lot less complex than C++. It's the most expressive typed language on the JVM, so if you like to think in types & you're on the JVM it's your best option.
Clojure is untyped, I hear many people praising it but I don't know any big project done in Clojure. So if you're doing short-lived projects I'm sure it can shine but for software that will be around for more than 5 years I would stay away from it. Btw, if misused, just like Scala, Clojure code can be extremely cryptic.
Go likes it's superficial simplicity, syntactic irregularity & stubbornly refuses to accept that PL design has evolved since the 80-90ies, but I'm sure it's appealing to people who are used to languages from that era.
Go is a language that is a bit tedious to write, no doubt about that, but it's very easy to read. I spend a lot of my time reading other people's code and I really appreciate that.
The fact that Scala as a language allows something like SBT to not only be created, but accepted, means I don't want anything to do with it.
I've suffered long from the Ruby ecosystem's mentality of "look at what I can do!" of self-serving pointless DSL's and frameworks and solemnly swore to myself to stay away from cute languages that encourage bored devs to get "creative".
It's about trade-offs, I guess. Go definitely appeals to a lot of people, and not all of us are unaware of the amazing "progress" that has been made in the 80's and 90's. Awesome progress that brought us Java, SOAP, C++, Javascript-on-the-server, and a slew of other tech some of us want to stay far, far away from.
> Go is a language that is a bit tedious to write, no doubt about that, but it's very easy to read. I spend a lot of my time reading other people's code and I really appreciate that.
Figuring out 1000 lines of code that could have been 10 and verbosity caused by a lack of generics is not going to help you understand code quicker. Figuring out what 10 lines of Scala do may take more time compared to 10 lines of go, but that's not a measure of velocity, the information density of go is just too low. At least 10 lines of scala fit on my screen, 1000 lines of go don't.
Code style issues imho are a team issue, if you do reviews these issues can be managed.
> not all of us are unaware of the amazing "progress"
Look at Rust, at least they did their homework. With Rust out there I can't see any reason to use Go except maybe their crappy GC.
I like Rust too. I re-wrote a parser I had implemented in Go in Rust and got almost an 8x speed-up.
Having said that, the two almost have no overlap for me. I don't see how Rust replaces Go in 9/10 of Go use-cases. And vice-versa.
> Awesome progress that brought us Java, SOAP, C++, Javascript-on-the-server, and a slew of other tech some of us want to stay far, far away from.
When people talk about progress from PL design, they aren't talking about any of those things. If you notice, all of those things were made in industry, not in PL research/academia. Not to mention that those languages are also ones that ignored the PL design progress! (although C++ finally seems to be adding some ideas from PL research in C++17)
They're talking about things like parametric polymorphism, dependent types, modules, macros etc. (I've mostly been reading about work in the types/ML family languages, but I'm sure there's progress been made outside that as well)
> The way you describe your experience with Scala makes me think you only had a very superficial look at it.
I worked with it daily for 2 years and also took Odersky's Coursera courses on Scala. I liked it more than Java (7), though I find the syntax aesthetically offensive (and I realize that's subjective). Ada (I worked in an Ada shop for 3 years before moving to the JVM) managed to have a robust type system without introducing the sort of syntax wtf-ness that Scala seems to need.
I actually recommended against using Scala at a later job simply because I thought the learning curve would be beyond most of the people I was working with (I didn't phrase it quite like that when mgmt asked me for my opinion, of course). Learning to write idiomatic Scala takes time. In that regard it's an expensive language to use unless you hire folks who already have experience with it.
As far as Clojure's dynamic typing goes, you can use libraries like plumatic/schema to add some checking where you need it (interfaces, etc), and for whatever reason I tend to have less trouble (as far as runtime type issues go) with Clojure than I do with Python or Ruby.
But hey, no language is perfect. I kind of wish Haskell clicked for me the way LISP seems to, since on paper it seems to check all the boxes -- but I just can't seem to get very proficient with it (or I'm just not willing to invest the time at this point).
> though I find the syntax aesthetically offensive
Well that's indeed your opinion, whenever I write code in a language with old fashioned statements I die a little inside. I have a lot of experience with ADA too, at it's core it's still a procedural language prohibiting good abstractions, it's very safe but also extremely verbose.
I learned Scala after I learned Haskell, ~7 years ago, maybe that's why I had a different experience. Since I learned haskell I think in types & transformations, it has made me a much better programmer. Clojure has sortof the same mindset, but I would call it 'shapes & transformations'.
> Clojure is untyped, I hear many people praising it but I don't know any big project done in Clojure.
Not sure your standard of big, but:
1. Most of Climate Corporation's (https://climate.com/) backend is written in clojure and they deal truly massive amounts of imaging data and parse/munge it to be useful for their applications.
>Go likes it's superficial simplicity, syntactic irregularity & stubbornly refuses to accept that PL design has evolved since the 80-90ies, but I'm sure it's appealing to people who are used to languages from that era.
Not really. The largest proportion of Golang users come from relatively modern interpreted languages like Python and Ruby which arr widely used in server environments. I'm unable to find the survey results but I do recall this trend surprised the Go's original authors who were originally seeking to replace C++ & Java
I guess Scala being "the most expressive typed language on the JVM" is true in the sense that it has a ton of features (OOP, FP, Exceptions, null Java backwards compatibility, etc.), but that's just too many features for a coherent language.
It's funny that both java & c# seem to be picking up many scala features in their latest & future versions like traits, lambdas, tuples, pattern matching, case classes, closed hierarchies, declarative generics...
Your language isn't incoherent if you can build features on top of each other with is exactly what scala does.
When I read comments like this I wonder if the person posting is trying to justify their own development decisions or seriously trying to "enlighten" the person they're responding to.
FWIW Python is also a great choice for composability. gevent is the only widely used library that really monkey-patches, and with it (to answer the question in OP) you can write essentially any function as a pseudo-goroutine (greenlets that yield to the event loop when reading from a queue which allows you to apply backpressure). And you can still access the vast realm of Python libraries; any that use sockets will automatically yield to the event loop on blocking operations. The criticisms from the recent Python complaint thread are valid, but it's still a great language for using other code.
For JS, rarely do people mutate Object.prototype these days; everything's a functional library. So almost all libraries you use will be good actors. It's also a good choice, though for data management and interop with scientific/vector/tensor operations, it's still hard to beat Python.
Interesting to hear. I like the idea of gradual typing as I'm coming from a predominantly python background and sometimes want to add types as I go. Perl6 looks really promising here. Curtis "Ovid" Poe has a good YouTube video on this. He starts with the Fibonacci function which can easily go wrong depending on a variety of inputs and keeps adding type restrictions such as it has to be a positive int between a range of #'s...i dunno, something along those lines.
I understand what they mean when they write:
I think the first time I appreciated the positive aspects of having a strong type system was with Scala. Personally, coming from a myriad of PHP silent errors and whimsical behavior, it felt quite empowering to have the confidence that, supported by type-checking and a few well-thought-out tests, my code was doing what it was meant to.
There are times when I appreciate the strict type-checking that happens in Java. I do get what they mean. But there are also a lot of times when I hate strict type-checking (in particular, when dealing with anything outside of the control of my code, such as whimsical, changing 3rd party APIs that I have to consume (for some business reason), or even 1st party APIs that feel like 3rd party APIs because they are developed by another team (within the same company) or for some reason we can not fix the broken aspects of some old API that was developed in-house 6 years ago.) Because of this, I have become a proponent of gradual typing. If I am facing a problem that I have never faced before, I like to start off without any types in my code, and then, as I understand the problem more, I like to add in more contract-enforcement. This is what I attempted to communicate in my essay "How ignorant am I, and how do I formally specify that in my code?" [1]
I think everyone who works with Clojure sometimes misses strict type-checking. Because of this, there have been several efforts to offer interesting hybrid approaches that attempt to offer the best of all worlds. There is Typed Clojure for those who want gradual typing, and there is more recently Spec. Given what I've written, you might think I am a huge fan of Typed Clojure, but I've actually never used it for anything serious. The annotations are a little bit heavy. I might use it in the future, but for now, I am most excited about Spec, which I think introduces some new ideas that are both exciting for Clojure, and which I think will eventually influence other languages as well.
Do watch the video "Agility & Robustness: Clojure spec" by Stuart Halloway. [2]
I also sort of understand what they mean when they write this:
No map, no flatMap, no fold, no generics, no inheritance… Do we miss them?
There are times when we all crave simple code. Many times I have had to re-write someone else's code, and this can be a very painful experience. There are many ways that other programmers (everyone who is not us, and who doesn't do things exactly like we do) can go wrong, from style issues such as bad variable names to deeper coding issues such as overuse of Patterns or using complex algorithms when a simple one would do. I get that.
All the same, I want to be productive. And to be productive in 2017 means relying on other people's code. And, in particular, it means being able to reliably rely on other people's code -- using other people's code should not be a painful experience. Therefore, for me, in 2017, one of the most important issues in programming is composability. How easy is it for me to compose your code with my code? That is a complex issue, but in general, those languages that allow for high levels of meta programming allow for high levels of composability. Both Ruby and Javascript and Clojure do well in this regard, though Ruby and Javascript both have some gotchas that I'd rather avoid. In all 3 languages, I find myself relying on lots of 3rd party libraries. I use mountains of other people's code. Most of the time, this is fairly painless. But there are some occasionally painful situations. With Ruby I run the risk that someone's monkeypatching will sabotage my work in ways so mysterious that it can take me a week to find the problem. And Javascript sometimes has the same problem when 3rd parties add things to prototype, perhaps using a name that I am also using. I so far have had an almost miraculous time using Clojure libraries without facing any problems from them. It's this issue of composability that makes me wary of Go. While I sometimes crave a language that simple, I can't bring myself to give up so much of modern languages best features.
[1] http://www.smashcompany.com/technology/how-ignorant-am-i-and...
[2] https://www.youtube.com/watch?v=VNTQ-M_uSo8