The big problem I had with Scala, and the reason I stopped actively developing in it months back, was its similarity to Perl's philosophy: there are too many ways to skin the same apple. Problems can be resolved functionally, imperatively, or in some (inevitably, when you come back to work on it, hard to follow) hybrid of the two; unless you set down clear restrictions and code guidelines, one man's Scala is entirely different to the next's.
I initially liked the expressivity the language afforded (and some of the supporting ecosystem - sbt, specs - I found extremely impressive), but came to abhor it over time. Unfortunately, after maintaining legacy Perl applications I've developed an aversion to anything that doesn't follow the Zen of Python: "There should be one-- and preferably only one --obvious way to do it."
> I've developed an aversion to anything that doesn't
> follow the Zen of Python: "There should be one-- and
> preferably only one --obvious way to do it."
this is an interesting argument. but surely, given enough rope, a non-expert can hang himself in any language. I'm starting to wonder that all these language X vs language Y problems are really a matter of the skills of the team matched with the difficulty of the problem. joe bloe's blog needs beginner skills with python. embedded wireless systems need experts and C/asm, nuclear control systems are prototyped by experts in haskell. bigcos need large teams of average skill with Java, but as they are driven to become more competitive, perhaps they will turn to small teams of experts using scala.
> surely, given enough rope, a non-expert can hang himself in any language
I completely agree, and sometimes productivity absolutely demands that developers become more expressive. I've found Java overly restrictive in the past, and have spent time learning Python, Haskell and other languages (and seen decent increases in productivity when I feel less fettered by the language I'm programming in). That said, I've also come to appreciate the effect of more restrictive languages: readability and long-term maintainability. Coming back to old code can always be jarring; having to maintain decade-old Perl written in a variety of different styles was maddening.
While you can screw up in anything, it's worth appreciating that more expressive languages effectively hand you more "rope" :).
It's easy to blame the user, but the fact is that there are languages designed in good taste and languages designed in bad taste. I'd argue C++ is an example of the latter, with its panoply of awkward non-orthogonal features. It's quite possible that Scala is too, and that it's not just the user's fault.
The difference is how easy a language makes it to hang ones self, or how many ropes are lying around to do it with. I think the article touches on it aptly. It's not so much a fault of the language (one can debate the semantics of it), so much as it is an outcome of greater-than n-sized (where n is 1, 2, maybe 3) non-high-expert-Scala-users teams. The lack of an easy to grok idiomatic way of writing code is hard on a team. It bothers me when I write code and don't know the idiomatic way of doing it, and I'll often solicit feedback and rewrite it just for my own sake (really, on code that no one else will ever really see or touch).
I spent a while in Perl and tend to agree, I was the only one in my group who went out of his way to write readable/maintainable code. Everyone else believed in extreme code ownership and enjoyed playing "golf".
I've never coded in Python, but spent a little time in Ruby and liked how they knew you could do things many different ways, so the community and leadership focused on identifying and promoting idioms. I think that's a key factor in adoption. Perl had some of that (and the excellent "Effective Perl" had a chapter on it), but you need to make it part of your culture.
> so the community and leadership focused on identifying and promoting idioms. I think that's a key factor in adoption. Perl had some of that (and the excellent "Effective Perl" had a chapter on it), but you need to make it part of your culture.
Yes it does need to be part of your culture. And there is no reason for a Perl team/shop not to because best practise idioms are heavily preached [1] and baked into [2] Perl and its community.
Python has been by far the most straightforward and easy language for me to pick up and just code and also read out of anything I can remember. Not that I use it a whole lot, it's still beautiful to me.
It does, but the point is that with Python there's generally one "obvious" way of accomplishing something. With Scala, I found when tackling certain problems (and my memory is fuzzy here; like I said, it's been several months :)), no particular solution stuck out as obvious. I'd consider myself fairly expert in Python now, so it may well be that what I see as obviousness may well be (reasonably) deep domain knowledge... but I didn't find the curve with Python anywhere near as steep as Scala's.
It's not nearly as ingrained into the language as Scala. Python's functional tools are rather basic. My understanding is that Guido believed that adding too much functional style to Python risked making the language muddled, exactly as we're discussing. A prime guideline for the language has always been (from the Zen of Python):
> There should be one-- and preferably only one --obvious way to do it.
has some notes on the history of functional tools in Python, though not much reasoning.
> Curiously, the map, filter, and reduce functions that originally motivated the introduction of lambda and other functional features have to a large extent been superseded by list comprehensions and generator expressions.
I was using Scala on some lone projects as prototypes, working with at most one other developer. In the end, we decided against it; there was never a chance to set down such guidelines (which I'm sure would be effective). The spectre of our Perl codebase was too much, sadly.
> I've developed an aversion to anything that doesn't follow the Zen of Python: "There should be one-- and preferably only one --obvious way to do it."
ZoP doesn't say that there should be only one way to do something, it say there should be one obvious way to do it. That doesn't preclude alternate, equally or - preferably - less obvious ways. Many languages could stand close to that aphorism.
There's something primal about Java that invites people to write it in styles that ape other languages. In fact, whenever I write Java I find myself trying to write FORTRAN or Lisp or something else. I did a stint of work in Scala that now has me writing Scala in Java. I'm starting to feel that Java is a better Scala than Scala!
It all started when I had to write Java classes that consumed results from Scala. There are all these Scala classes like Some that, if you look at the docs, you realize you can use in Java without any trouble at all.
Many of the patterns that Scala uses can be implemented easily in Java... If there's something you like about Scala, you can usually do it in Java with a little thinking.
A good example is all of that Actor BS in Scala. You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how. Java's got ExecutorService, which for many of the tasks I do, can keep an 8-CPU machine busy with 100% utilization.
I remember having a discussion with a guy in Germany about his system that used Scala Actors that revealed that neither of us had any idea of what was really going on at all. Today I build pure Java systems using Java idioms that are so fast it's almost scary.
"You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how"
I think this was the turning point for me, the day I realized someone had created this large concurrent system with absolutely no idea or care as to what was going on. (the not uncoincidentally same day qa discovered some serious problems making the app become almost useless with extremely high CPU loads)
I've been playing around inside the closure-compiler in my free time recently and the thing that struck me most after years of working in other languages (Ruby, JavaScript, C#) is that Java is relatively simple. While it's obviously a very small sample set, I have yet to find myself spending any significant amount of time wondering at what a given snippet of code does.
I don't have any Scala experience but in general simplicity has major and obvious benefits (as noted many times in the letter).
Yes, Java is generally fairly simple. I generally find that the most difficult part of Java is generics.
On the other hand, anything that you want to work at a slightly higher level, such as closures or higher-order functions, Java just gets in your way since you have to it all by hand.
Thinking about Scala, there are a couple of problems with its simple/complexity balance:
1. Some things that appear fairly simple actually explode into a huge mess of complexity under the hood. Often, that's not a problem. However, sometimes it becomes a problem because you don't really know what's going on.
2. Some things that should be simple and straightforward, like iterating through an array, are impossible. These are cases were the language is actually removing some of the power of the underlying platform.
You're right, it's not quite impossible. However, the idiomatic ways of doing things in Scala tend to be very bad for performance if you're in a tight loop. For example, if you have two arrays that you need to iterate through simultaneously, you need to zip them together. In order to regain the performance, you end up having to do something like the while loop or use recursion. This is similar to the problem of doing heavily numeric code in Clojure before release 1.3. You could get Java-like numeric performance out of Clojure, but you'd have to abandon doing it in any sort of ideal way.
One example where something was impossible in Scala is that to implement a Parcelable for Android. This requires the creation of a static field, which Scala doesn't allow. Now, I won't argue that an API that requires such a construct is good, but it might be necessary for your application. I believe Scala now contains special code in the compiler just to support this peculiarity in the Android API.
In your example of iterating through two arrays, the proper Scala idiom is (a1,a2).zipped.map((v1,v2) => ...). This avoids doing an actual zip which would have significant overhead. It also makes it easier to write the map as the map method on Tuple2.Zipped takes a function of two arguments so you don't have to deal with pulling values out of a tuple.
I would actually find the details of where Scala performance is poor to be very interesting. Most significant IMO would be whether they are real problems or people picking improper implementations. For example I'd like to be able to answer the following, would using a view have given them the performance they needed without producing ugly code?
Isn't the counter-argument that Java code is harder to understand at a slightly higher level, of how classes interact? And that in more expressive languages, that complexity is pushed down to the code level, so the total complexity is the same, just in different places.
Of course, there's an advantage to being able to understand a snippet locally, in isolation from the rest of the project, if that's all you need to understand (e.g. when making a local change).
Do you mean that, because code is more expressive/terse, less project level (class, factory, etc) juggling is required? What you're saying seems to make sense but I'm trying to conjure an example to help me understand it.
Yes. There's a family of examples with design patterns, where what is a design pattern in a less expressive language like Java requiring a standard arrangement of classes, is absorbed into the language in a more expressive language. The "design pattern" is a kind of redundant overhead. Describing examples is a bit involved, because you have to describe both the pattern and the expressive alternative.
One concrete example is the visitor pattern, which is more simply written with multi-methods (a multi-method is one that is dynamically selected by the type of its arguments - though Java can have methods with the same name but different arguments, they are selected statically, based on compile-time types rather than runtime classes).
I actually can't think of any other examples off the top of my head, but "design patterns are language features in more expressive languages" is a common idea, so I'm sure others can suggest other examples (or google).
I actually can't think of any other examples off the top of my head, but the "design patterns are language features in more expressive languages" is a common idea, so I'm sure others can suggest other examples (or google).
As a specific case, Python has many design patterns built into the language. Iterator pattern: generators and iter/next. Factory pattern: have class objects override __new__, callers don't need to know the details. Singleton: just use module-level functions; modules are themselves objects if you need to replace/stub/mock them. And as with any language that has first class functions/methods/callables, the command and strategy patterns mostly go away.
That's another good one. I think people tend to apply this principle to concepts from functional languages, but forget that there are many abstractions in imperative languages that simplify styles of programming in the same way.
That's one of the main points of pg's "Beating the Averages" essay, where most people got the term "Blub programmer": http://www.paulgraham.com/avg.html
The visitor pattern is a good example, but I think another good one are classes. You can write object-oriented programs in C, but it requires that you, the programmer, enforce all of the concepts of classes (member functions, instantiation, inheritance, etc.) yourself. I think that is a clear example where more expressive/terse code (that is, language-level support for OO) requires significantly less juggling across the code base.
@codahale, was this email from you or someone else in your company?
Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime.
As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java!
That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow!
This means that, if you want amazing performance in Scala, you have to write C-like code in Scala as well. I'd expect anyone who needs top-notch performance to know this kind of thing already. Writing performant code in any high-level language in our day and age, even C++, still requires you to understand how your abstractions are actually getting implemented underneath the hood. Scala doesn't work miracles here or everyone would probably be using it already.
Just ask the people who run JGit about performant JVM code. (http://marc.info/?l=git&m=124111702609723&w=2) They ran into a ton of cases where they couldn't write the equivalent C-code in Java and performance suffered as a consequence.
However, the points about the build toolchain, backwards binary compatibility, and to some extent, initial learning complexity are the ones that I think are most damning against Scala. I hope Typesafe has a good response to this or other companies step up to share how they worked around these deficiencies successfully. Otherwise, I feel like this could be potentially very damaging to Scala's reputation, and maybe even deservedly so.
I've found Scala to be a fabulous language for writing > 90% of my app(s). The overhead of Scala's built-in collections, for loops, property accessors/mutators, closures, etc is simply unimportant outside of critical hotspots and provides so much in terms of brevity and expressiveness.
Yes, Scala's lack of looping constructs (break, continue, a real indexed for loop) means that you're better off writing performance-critical things like crypto or protocol parsing code in Java (or if it really matters, C called via JNI). But once you're concerned about performance at that level, every language requires you to carefully consider what your code is really generating, what your allocation patterns are, etc.
Scala is a much more expressive language compared to Java, and the Scala standard library is excellent. Completely dropping Scala for Java when you're building large, complex systems seems like chopping your foot off because your small toe hurts. Use Java or C for the % of code that needs it and Scala for the rest!
The complaints were not primarily about performance. The bigger issue seemed to me the difficulty of finding people experienced in Scala and teaching it to those who are not, (due to the complexity of the language), and the difficulty of producing maintainable code (due to frequent changes in the language and a lack of standard practices).
Expressiveness alone does not make a language suited for building large, complex systems.
Part of the original complaint was the issues introduced by using a combination of Java and Scala rather than just one or the other (with Scala-only being impractical due to the need to use Java-based libraries).
"2. Don't ever use scala.collection.mutable. Replacing a scala.collection.mutable.HashMap with a java.util.HashMap in a wrapper produced an order-of-magnitude performance benefit for one of these loops. Again, this led to some heinous code as any of its methods which took a Builder or CanBuildFrom would immediately land us with a mutable.HashMap. (We ended up using explicit external iterators and a while-loop, too.)"
Overall Scala<->Java interop is pretty seamless, and aside from some annoying things (for loop becomes foreach with a closure, break & continue are done with exceptions, etc) it's quite easy to treat Scala as a more concise Java with very similar performance.
Rather than using a Scala collections wrapper for the java.util.HashMap he should have simply used the Java HashMap as-is. Yes you must do hashMap.get("key") instead of Scala's shortcut hashMap("key"), but so what? You're talking performance-critical code already, wouldn't you want to avoid the wrapper?
"avoiding generics like the plague, including java.util.HashMap"
With generics being a compile time concept, you probably meant "avoiding collections like the plague"
Promises have been made in the comments, things did either not get better or even got worse since then (Collection API, the nightmare that is SBT 0.1X, binary compatibility).
Interesting read. Honestly, Scala looks like gibberish to me. I strongly distrust a language where I can't hold its conceptual model in my head all at once. This is true for C, true for JS, true for ML, and hell even true for Common Lisp (once you realize it's all macros over a simple core). It is not true for Scala or Haskell, at least for me.
> As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.
The power continuum isn't the same as the conceptual simplicity continuum. C++, for example, is less powerful than Common Lisp while being more complex as well.
At the same time, is Haskell more powerful? Common Lisp, relative to C++, buys me the ability to run arbitrary functions at compile-time and generate code. Haskell takes away that power, and as far as I can tell doesn't add anything to replace it. Indeed, it lets your code do strictly less thanks to the type system.
There's also the reliability/error-proneness continuum.
Haskell is probably less powerful than Common Lisp (Template Haskell technically yields the power, but it is less practical). The type system may also rule out some useful programs.
However, Haskell yields more reliable, less error-prone programs. The number of runtime bugs that get through the Haskell compiler is substantially lower than that of Common Lisp (I venture this is still true if you use CL's optional typing).
Haskell also has forms of power that as far as I know, Common Lisp does not have. Namely, type-classes allow expressing various forms of static-type-based polymorphism that's not expressible via Common Lisp (e.g: Return-type polymorphism).
In practice, having a smaller emphasis on macros pushed Haskell to abstract various repeated patterns using means other than macros. Some cases are covered by laziness. Others are covered by passing around simple functions. Some are covered by special wrapper types and combinators.
My personal opinion is that the extra composability of these alternatives is more beneficial than their weaker power is detrimental. The cases where alternatives to macros cannot be found are relatively rare, so the extra cost of Haskell's macro system (TH) rarely needs to be paid.
I wrote a medium sized project in clojure at my last job and it was a good experience. The team was small (under 5) so the hiring and training wasn't an issue for us. I never had a problem reading clojure code, and the only major headache I can remember, which was our fault, was that clojure keywords (think ruby symbols) used to be turned into interned strings which are stored in permgen and we were running out of permgen space. I believe this has been fixed in recent versions of clojure, but I'm not 100% sure about that.
So does Erlang, but it's not really a bug. Symbols (atoms, whatever) are guaranteed to be a unique identifier, so the fact that they're allocated and exist for the duration of execution isn't really surprising.
So I'm not a big lisp guy, but from what I understand and what I know of CL, lisp is in a much better place on the expressiveness:simplicity relationship than Scala.
That said, some of these issues (slow collections, talent availability, tools) have nothing to do with language complexity and might affect almost any relatively new programming language if the implementors didn't consciously work to counteract it.
Lisp is in a very different domain, being dynamically/uni-typed, whereas Scala is statically typed. Statically typed languages don't aim only for simplicity/power ratio, but also for various compile-time guarantees.
"That's just conpletely false. I don't know where your assertions are coming from but language design absolutely has a great deal to do with what kind of tools / developers you can expect to find to support the language. I'm all a-huff with the sheer gall and ignorance of your assertion that I'm just not even sure what to do. Bad developer. >:( bad"
Revised: fine ok maybe a bit harsh. My opinion is that language design would be extremely influencial over how people implementing collections/tools/compilers go about doing their work and what the end result of their efforts would look like. In fact they probably are more telling of a design than anything else. That last part was sort of bs guesswork but it sounds right. Happy now?
I look at the talent availability problem as an HR / Mangement issue. It feels like they want to do checklist hiring instead of hiring great developers who can learn. I would rather take the time needed in hiring, but at some point it becomes gears in the machine.
Can you elaborate? Do you mean that the criticism is valid (which I agree is), or that you agree with the conclusion of the detriments outweighing the benefits?
While I haven't used Scala on an "enterprise" scale, I can certainly agree with many of the criticisms. However, in my experience, Scala is still worth it.
I don't deal with training a team of developers, nor do I have to integrate with enterprise-scale infrastructure, so those things just don't apply to me.
What are the benefits of Scala? I can think of a few, off-hand:
1. Traits: I really like traits and the way that you can use them to encapsulate cohesive and reusable functionality. In some ways, traits allow you to do a form of aspect-oriented programming.
2. Case classes are a nice, simple way of creating basic data structures. In Java, creating a data structure that should have a proper equality implementation and a nice toString is a tedious chore. Granted, I think Clojures record's are better since they give you immutability, too.
3. Closures. Other languages also have them, but Java does not. Being able to use closures can really help deal with things such as handling events. In Java, you have to resort to creating inner classes, some of which end up instantiated as members of the class.
4. Judicious use of implicits can give you some half-decent DSLs.
5. 3+4: A nice implicit conversion of a one- or two- line block into an appropriate instance of a ManagerAdapterEventListener can really simplifiy the code.
6. The mixed paradigm approach can be a blessing sometimes. I do prefer the functional approach. However, if you have integrate with a heavily object-oriented framework, being able to be object-oriented can be helpful.
Overall, I would generally prefer Scala over Java for my projects. However, I do prefer Clojure over Scala.
I can certainly agree with many of the
criticisms. However, in my experience, Scala
is still worth it
Considering the alternatives, C++ was and is still worth it. However because of its popularity, it prevented and still prevents other languages from gaining critical mass, making it the only game in town, with the sole exception being Obj-C, but that's only because Cocoa was designed for it and Apple doesn't mind a little technology lockin either.
When's the last time a development team making an internal technical decision became something a company chose to "officially announce"?
"ÜberCorp announced to shareholders today that it decided once and for all curly braces belong on a new line. And that only fumbling retards would choose vi over emacs.."
This episode illustrates the reason to keep important communications polite and well argued, you just never know who the eventual audience might be. I think the author expressed himself well enough that this shouldn't be too embarrassing.
Be that as it may, I'm quite grateful to whoever did the original writeup. He did a great job of doing it respectfully and providing a ton of good data for people considering Scala, or really any new language.
Yeah. Coda's a good guy, and I'm not saying otherwise.
What I'm saying is that Stephen Colebourne is being a dick, because he used some leaked email as some sort of "justification" for his poorly argued "attacks" on Scala.
His whole post is a "I told you so! Look!" even though his original argument had no technical merit what so ever, and pales in comparison to the sort of constructive criticism that Coda is providing to Typesafe in the leaked email.
I mean, half this email is about performance/micro-opts. These things aren't really unknowns to anyone using Scala in a production setting.
FWIW, we're using Scala at a bigger scale than Yammer, and yeah, there's some things that aren't great, but on the whole, using Scala is a joy compared to straight Java. We also have no issues with sbt or using the latest Scala (we're on 2.9.1, it was a fairly easy upgrade from 2.8. Our 2.7 -> 2.8 upgrade was more a pain in the ass. :) Our only major issue is insanely long compile times.
This whole thing is amazingly pointless, because no one is saying Scala is magical and without room for improvement (no one I know, anyway). Constructive criticism is great, but there's nothing here that hasn't been said on the Scala mailing lists.
The real shame about this whole thing is how much respect Stephen has lost in a lot of people's eyes. Joda Time is the best library for date/time manipulation on any platform/language. I love it. But now I have this irrational compulsion to rewrite the whole thing because I want nothing to do with Stephen. Not because I disagree with him, but because I think he's gone about everything entirely the wrong way.
It is interesting to see how start-ups (twitter, forsquare and now yammer) adopting or dropping programming languages rains hailstorm on twitter, HN and blogosphere. As a serious PL designer, start-ups seem to be the prime market to target your language.
I found Neal Gafter's response to Stephen's blog rather telling:
blog.joda.org/2011/11/real-life-scala-feedback-from-yammer.html?showComment=1322613312501&m=1#c6062812671177626424
I initially liked the expressivity the language afforded (and some of the supporting ecosystem - sbt, specs - I found extremely impressive), but came to abhor it over time. Unfortunately, after maintaining legacy Perl applications I've developed an aversion to anything that doesn't follow the Zen of Python: "There should be one-- and preferably only one --obvious way to do it."