Let's just say that I have a bit more, up-to-date experience with the language and I value its consistency, coherence and elegance.
One thing I really like is that Scala pushes for more general, generic solutions, instead of ad-hoc additions and hacks: implicits instead of extension methods, traits (instead of abstract classes + defender methods), objects (instead of “static” members), types (without arbitrary rules about what is allowed as a type and what not), pattern matching via apply/unapply, for comprehensions via map/flatMap/withFilter, methods instead of methods and properties.
That tons of languages (Java, Kotlin, Ceylon, ...) are copying Scala's design decisions (often badly, but nevertheless) is another sign that Scala got a lot right.
Ok ... whatever, if I have already written so much, I can just answer to your claims one by one (I hope that the time I spend on this will at least be slightly appreciated):
PART ONE (Hackernews complains that it is too long)
what is the problem Scala is trying to solve
Being a modern, typed language which gives people the right tools to solve today's and tomorrow's engineering requirements.
I know that Erlang and Clojure try to solve the problem
of writing concurrent code (and fault-tolerant code in
Erlang's case).
Scala fixes a some issues of Erlang's design and improves on it in a few substantial areas (which can't be fixed in Erlang itself anymore due to backward compatibility).
It has better performance and better monitoring support.
Additionally, it offers better and more diverse tools to tackle concurrency than Clojure.
Haskell tries to make writing correct code easier.
While Scala does not enforce purity by default (there is an effect system plugin for that) it gets you a long way towards Haskell's “if it compiles it is most likely correct” guarantees.
Ruby and Python were made for ease and productivity
Apart from the “batteries included” approach (Scala prefers a minimal standard library instead¹) my experience is that it can easily match or beat Rupy's or Python's productivity.
¹ It also provides better tools to fetch additional dependencies than the languages mentioned above.
both Ruby and Clojure are great for DSLs
Well, people say that about Scala, too. I don't see the big deal about DSLs, I just try to design and implement the best API a library can possibly have and Scala gives me the right tools to make that happen.
Java and C are used nowadays for performance
Scala can match and beat Java's performance (looping seems to be faster than in Java, but I never understood why, optimization, specialization, macros, ...).
Java is relatively good for architecting huge software
systems
Scala's better OO and module support improves on that.
Now that's great, and Kotlin is all that, too.
Kotlin is a train-wreck. They promised a lot of things, but failed to deliver on pretty much everything. Sadly, those parts which were not just direct copies of Scala's design show the lack of experience in language design.
I think it is pretty ironic how their beloved talking points about “why not Scala?” has been reduced to almost nothing as they have continued to learn why Scala did things in a certain way. Just compare one of their first presentations with one of their latest ones.
They should really stop talking and start shipping if they want to be taken serious, because as a paying JetBrains customer I'm getting really tired of their vaporware and FUD.
Why the immutable data-structures, then? To make
concurrency better?
Partially. It makes reasoning about the program much easier in general and allows safe, structural sharing of data.
In that case, why is mutability just as easy?
Because Scala is not Haskell. Scala gives you tools to get your job done, it doesn't require you to adopt some ideology or religion.
Sometimes, a mutable algorithm/data structure fits a requirement exactly and Scala won't annoy you for picking it.
And what are implicits
Generally speaking, implicits are a generic way to guide the compiler towards closing a gap. What's such a gap?
It can make existing types implement new interfaces (think arrays, java.lang.String, ...), it wires up type class instances with methods which require them, it can make incompatible things compatible (e. g. types which come from different third-party Java libraries).
Have a look at how String is made to support Scala's collection API.
Have a look how the `to` method in Scala's collection library can work with arbitrary collection types (which don't even need to be known to the standard library).
They wouldn't be necessary in a perfect world, but Scala is pragmatic language and its designers acknowledge that we are not living in a perfect world. The cost/benefit ratio of implicits compared to things like extension methods is magnitudes better.
and these new cringe-inducing macros for?
They provide a general way to make APIs more safe and implementations more efficient. They can be used to report more specialized errors right at compile times, they can be used to make sure that your closures don't close over things you don't want, they can be used to implement LINQ to query databases while using the bog-standard collection API, they can be used to implement F#'s type providers. This can all be done with full type-checking and refactoring support from the IDE/compiler instead of having to resort to such terrible things as annotation processors, bytecode rewriting and Java agents.
They are a huge improvement over Java's approach and Oracle is now copying parts of it.
DSLs? Why would a high-performance, statically typed
language make it easy to write DSLs?
Why not? Just because it is a DSL, it doesn’t mean it has to such on the performance/safety front.
Why all the OOP, then?
Because OO is a good tool to solve some problems, just like FP is a tool to solve some other requirements.
Oh, it's to combine the too; in that case why do they
feel so strenuously glued together
I think you have to be more precise here. Even people coming from OCaml or F# concede that Scala has done an incredibly good job at combining OO and FP, so I'm happy to hear what issues you have found.
(classes vs. case classed,
Well, for some things it make sense to have the additional methods of a case class, for some use-cases it doesn't.
an entire collection package replicated twice, once for
the mutable case and once for the immutable).
Pick the best tool for your job. Some algorithms work best with immutable data-structures, some with mutable. Scala spells out explicitly which guarantees are made and people can safely rely on it.
Experience has shown that Java's approach had good intentions but just didn't work. Even the designers of Java agree with that these days. Scala has learned from those mistakes and doesn't repeat them (unlike Kotlin).
So the language offers a powerful compiler but absolutely
no guidance on how a program should be written.
That has not been my experience. There is some local immutable-OO-with-FP-with-typeclasses optimum and people regardless of where they come from are almost magically converging towards it.
If at least Scala had somehow provided all of these
features and stayed elegant,
I think it does.
but man, it would take you weeks just to understand how a
freaking Scala collection works, just because the
language designers wanted to be so clever and prove that
you could transform collections and still never require
casts.
Well, it's a bit more than that, right? Anyway, I think everyone in the Scala space is open towards a better solution, but frankly even after years no other language has come up with an approach which comes even close to collection API's ease of use.
It seems that at every turn Scala favors cleverness over
clarity
In my experience, readability and clarity are considered more important these days. Cleverness is deemed to be OK if it is used to improve the lives of people using that piece of API. It's just like mutability: It's ok as long as you keep it localized, confined and don't unnecessarily expose your users to it.
features over a cohesive philosophy
I think I disagree with that. Consistency is still one of the most important requirement and I don't have seen much features to make it in the last versions.
Anyway, Scala has much less features than Java 8, C#, F# and many other “competitors” in that space, so I think we are fine here.
Scala chooses, over and over, to try and address more
goals (most are really unimportant), and in the process
has lost the most important thing in a language:
coherence.
As mentioned, this has not been my experience, but I'd love to see an example.
Scala sees something nice in another language and
immediately adopts it.
No, absolutely not.
And I gotta say, writing a compiler that compiles code
that's both javascript and Haskell is an impressive feat
of engineering. But it comes at such high a price...
One thing I really like is that Scala pushes for more general, generic solutions, instead of ad-hoc additions and hacks: implicits instead of extension methods, traits (instead of abstract classes + defender methods), objects (instead of “static” members), types (without arbitrary rules about what is allowed as a type and what not), pattern matching via apply/unapply, for comprehensions via map/flatMap/withFilter, methods instead of methods and properties.
That tons of languages (Java, Kotlin, Ceylon, ...) are copying Scala's design decisions (often badly, but nevertheless) is another sign that Scala got a lot right.
Ok ... whatever, if I have already written so much, I can just answer to your claims one by one (I hope that the time I spend on this will at least be slightly appreciated):
PART ONE (Hackernews complains that it is too long)
Being a modern, typed language which gives people the right tools to solve today's and tomorrow's engineering requirements. Scala fixes a some issues of Erlang's design and improves on it in a few substantial areas (which can't be fixed in Erlang itself anymore due to backward compatibility). It has better performance and better monitoring support.Additionally, it offers better and more diverse tools to tackle concurrency than Clojure.
While Scala does not enforce purity by default (there is an effect system plugin for that) it gets you a long way towards Haskell's “if it compiles it is most likely correct” guarantees. Apart from the “batteries included” approach (Scala prefers a minimal standard library instead¹) my experience is that it can easily match or beat Rupy's or Python's productivity. ¹ It also provides better tools to fetch additional dependencies than the languages mentioned above. Well, people say that about Scala, too. I don't see the big deal about DSLs, I just try to design and implement the best API a library can possibly have and Scala gives me the right tools to make that happen. Scala can match and beat Java's performance (looping seems to be faster than in Java, but I never understood why, optimization, specialization, macros, ...). Scala's better OO and module support improves on that. Kotlin is a train-wreck. They promised a lot of things, but failed to deliver on pretty much everything. Sadly, those parts which were not just direct copies of Scala's design show the lack of experience in language design.I think it is pretty ironic how their beloved talking points about “why not Scala?” has been reduced to almost nothing as they have continued to learn why Scala did things in a certain way. Just compare one of their first presentations with one of their latest ones.
They should really stop talking and start shipping if they want to be taken serious, because as a paying JetBrains customer I'm getting really tired of their vaporware and FUD.
Partially. It makes reasoning about the program much easier in general and allows safe, structural sharing of data. Because Scala is not Haskell. Scala gives you tools to get your job done, it doesn't require you to adopt some ideology or religion. Sometimes, a mutable algorithm/data structure fits a requirement exactly and Scala won't annoy you for picking it. Generally speaking, implicits are a generic way to guide the compiler towards closing a gap. What's such a gap? It can make existing types implement new interfaces (think arrays, java.lang.String, ...), it wires up type class instances with methods which require them, it can make incompatible things compatible (e. g. types which come from different third-party Java libraries).Have a look at how String is made to support Scala's collection API. Have a look how the `to` method in Scala's collection library can work with arbitrary collection types (which don't even need to be known to the standard library).
They wouldn't be necessary in a perfect world, but Scala is pragmatic language and its designers acknowledge that we are not living in a perfect world. The cost/benefit ratio of implicits compared to things like extension methods is magnitudes better.
They provide a general way to make APIs more safe and implementations more efficient. They can be used to report more specialized errors right at compile times, they can be used to make sure that your closures don't close over things you don't want, they can be used to implement LINQ to query databases while using the bog-standard collection API, they can be used to implement F#'s type providers. This can all be done with full type-checking and refactoring support from the IDE/compiler instead of having to resort to such terrible things as annotation processors, bytecode rewriting and Java agents.They are a huge improvement over Java's approach and Oracle is now copying parts of it.
Why not? Just because it is a DSL, it doesn’t mean it has to such on the performance/safety front. Because OO is a good tool to solve some problems, just like FP is a tool to solve some other requirements. I think you have to be more precise here. Even people coming from OCaml or F# concede that Scala has done an incredibly good job at combining OO and FP, so I'm happy to hear what issues you have found. Well, for some things it make sense to have the additional methods of a case class, for some use-cases it doesn't. Pick the best tool for your job. Some algorithms work best with immutable data-structures, some with mutable. Scala spells out explicitly which guarantees are made and people can safely rely on it. Experience has shown that Java's approach had good intentions but just didn't work. Even the designers of Java agree with that these days. Scala has learned from those mistakes and doesn't repeat them (unlike Kotlin). That has not been my experience. There is some local immutable-OO-with-FP-with-typeclasses optimum and people regardless of where they come from are almost magically converging towards it. I think it does. Well, it's a bit more than that, right? Anyway, I think everyone in the Scala space is open towards a better solution, but frankly even after years no other language has come up with an approach which comes even close to collection API's ease of use. In my experience, readability and clarity are considered more important these days. Cleverness is deemed to be OK if it is used to improve the lives of people using that piece of API. It's just like mutability: It's ok as long as you keep it localized, confined and don't unnecessarily expose your users to it. I think I disagree with that. Consistency is still one of the most important requirement and I don't have seen much features to make it in the last versions.Anyway, Scala has much less features than Java 8, C#, F# and many other “competitors” in that space, so I think we are fine here.
As mentioned, this has not been my experience, but I'd love to see an example. No, absolutely not. Huh? That doesn't make sense.