I would never say that Scala's designers are stupid. Far from it. The Scala compiler is a work of brilliance. And, obviously, every feature, as you so meticulously tried to present, has a purpose; tries to solve a problem. But your explanations, I feel, only prove my point. Many of your explanations are along the lines of because sometimes you need that ("Some algorithms work best", "a good tool to solve some problems", "for some things it make sense"...). While absolutely true, and every practical language, be it a programming language or a spoken language, needs versatility and irregular forms, it seems like Scala tries to take on each one at a time rather than spending most of the intellectual effort on defining the common case.

For example -- and this is an important philosophical point of disagreement -- you say of macros: "... instead of having to resort to such terrible things as annotation processors, bytecode rewriting and Java agents." This, imho, the WRONG answer. Those problem areas that in Java are addressed by what you call "terrible" means, are highly irregular; highly uncommon. They should be addressed by "terrible" means if your goal is a simple language. Yet Scala seems to want to address every problem with a language feature, and in this case it's a huge one.

On the other hand, when I look at Erlang and Clojure (or Ruby, though I'm less familiar with it), I see languages that were designed by people who sat down and thought long and hard about which are the top one, two or three most burning problems of software development, and then tackled those and only those. Everything else would be solved possibly "terribly" (though it would be solved). Rich Hickey thought long and hard and came to the conclusion that while OOP might be the right solution sometimes, in the end it's more trouble than it's worth, and people should not generally use it to write most programs. He may be wrong, and he may be right, but he made a decision. He thinks (I guess) that if your particular problem absolutely requires OOP, then you're better off using a different language, one that's been lovingly crafted for that purpose.

This is extremely important. A coherent language says, "for my problem domain these are the tools you should use". A general-purpose coherent language adds "... and most problems fall in this problem domain". For whatever is left, use other, better suited languages. Scala never says this. For every problem, big and small, it tries to find a solution in Scala. I mean, it's running on the JVM for god's sake, and interoperability on the JVM is particularly easy. Why not come out and say, "DSLs are great; we absolutely love them; if you want to use them, write them in, say, Groovy"?

I did not intend to ask why would you ever need this feature or that? What I asked was, why must they all be in the same language? If you had said, "look, Scala just tried to do this, but because of sheer genius it just happens to do that, too", that would have been a good answer. But you didn't. Each feature is there to solve a different problem. That's why Scala lacks coherence.

A non-coherent language says, "here are the tools you can use". It says, "in your programming endeavors you might some day encounter this byzantine problem, and guess what? We got a tool just for that!" It lays out a huge set of tools, all usable at some point or another, all serving a purpose, but doesn't say "I think you should rarely use this tool or that, so I'm leaving them out of the toolbox. When you need them, buy them separately" (Worst of all, it gives you a bulldozer when all you need is a hammer. That's why it's unwieldy)

These are two competing philosophies, but for modern software development, the latter is the wrong choice and the former is the right one. Software systems are getting bigger and more complicated as it is, while programmers aren't getting smarter. Some challenges are much more important than others. Scala chooses to be Jack of all trades and master of none[1] in the very discipline that needs the opposite approach.

[1] What is the one or two things Scala is better at than any other language? For Clojure it's managing state; for Erlang it's fault tolerance. Both are at the very top in some other aspects as well. But what does Scala do better than anyone else? (and is that thing important? You might say it's best at marrying OOP and FP -- though even if that's the case I'd say being best doesn't mean you're good enough -- but I don't think that anyone would say that combining these two paradigms is what the software industry needs most. Or, you might say, typed OOP. But typed OOP is, again, a compiler feature, not a solution to a burning problem)

Here are the two things why I am using Scala:

1. Confidence

Scala gives me the confidence that I can build software the way I imagine, I can focus on the user of my code, not on making the compiler happy.

While there are plenty of languages which make easy and medium problems nice to solve (Clojure and Erlang certainly belong to this group), Scala is one of the few languages which keeps supporting me regardless of whether my problem is easy, medium or incredibly hard.

In my experience, the work on making hard problems easier had a huge tricke-down effect which in turn improved Scala's issue solving capabilities for simple and medium issues.

I think the language is better for that and certainly ahead of Erlang and Clojure here.

While most hard problems are not common, they are often fundamental. Not being able to solve some issue in the best possible way can have huge negative impact on the whole application and library design. That's why for instance making it easier to create macros wasn't the first problem to concentrate on. Instead, developers made sure that users of macros had the best possible experience and focused on having one, unified API for reflection, the macros and the compiler, hugely simplifying semantics while re-using battle-proven code. Macros makes it possible to pull more functionality out of the language and out of the compiler; into libraries. For instance C# 5 introduced huge language changes with adding async/await to the language. In Scala no language change is necessary: Adding support for async/await would be just a library.

Unlike Macros in most other languages and inferior approaches like in Java, Scala macros are type-checked as regular Scala code at the definition site as well as the macro expansion at the call site, removing huge amounts of tricky issues all at once.

Great care is taken to make and keep Scala a highly regular language with only the minimal amount of hardcoding necessary to make things work. Unlike Clojure, it doesn't have special-cased syntax for collections and a few “blessed collection types” shipping with the language. Unlike Erlang, Actors are not built into the language.

In both cases, Scala avoids irregularity by enabling users to build libraries which can be improved and be replaced without much trouble.

  Yet Scala seems to want to address every problem with
  a language feature, and in this case it's a huge one.

  On the other hand, when I look at Erlang and Clojure 
  (or Ruby, though I'm less familiar with it), I see 
  languages that were designed by people who sat down 
  and thought long and hard about which are the top one, 
  two or three most burning problems of software 
  development, and then tackled those and only those.

Is it hard to build a language with these intentions? Sure! Is that a reason not to do it? Absolutely not. I think one part where Scala has basically proven tons of people wrong is OO/FP. People have been saying for decades that OO and FP are fundamentally opposed to each other. Scala just went ahead and proved them wrong, showing that just because some earlier approaches like OCaml or F# are not that good doesn't mean it is impossible. Also, people have been claiming that there will always be a impedance mismatch between languages and databases. Scala went ahead and showed that it has not to be that case.

I want the best OO functionality combined with the best FP functionality. I want to be able to use higher-order abstractions combined with the best performance and efficiency. I want libraries written in the best possible way, not in the way the language decided it was convenient. I want to use the right tools for the right job without having to migrate from one language to another.

  He may be wrong, and he may be right, but he made a decision.

Clojure or Erlang just don't deliver here and that Clojure is the best language to manage state is highly debatable, too.

  I think you should rarely use this tool or that, so I'm 
  leaving them out of the toolbox. When you need them, buy 
  them separately

Don't take me wrong, a language should be as easy as possible — but not easier.

2. Community

It is pretty non-sensical to ask “What is the one or two things Scala is better at than any other language?”. There plenty of things it does better, because “good enough” is just not good enough for people in the Scala community.

In general the Scala community is highly critical of every aspect and tend to push things to the current state of the art or beyond it if they feel something can be solved in a better way. This has led to a huge increase of consistency and quality throughout the ecosystem, so that having a few good parts and a lot of mediocre parts is just not acceptable to most Scala developers anymore. They demand the best tools one can possibly build.

Anyway, I think your use of “coherent” is getting more clear, but imho makes less and less sense. You are basically asking for a silver bullet and are unhappy that Scala tells you that for many problems, there isn't one. I think this is one of the core advantages of the community: It doesn't try to sell you some “ultimate solution”, avoids ideological bullshit and treats people as grown-ups.

For instance, Scala's Akka team (those who work on concurrency-related libraries) had an interesting talk recently where the demonstrated something like 9 different approaches/techniques to tackling concurrency, all of them with different benefits and drawbacks, with the main conclusion of “pick your poison”.

I think this is one of the core distinctions between Scala's diverse community and other, more anglo-saxon-centric communities: People who have grown up in the US just love to swallow shallow marketing non-sense and respond extremely well to claims about “one true way” or “silver bullets”.

If somebody came with that approach to the Scala world, people would tell him/her that he/she is either lacking experience, has poor judgement, or probably both and show him/her why he/she is wrong.

The way people carefully evaluate different approaches and document its pros and cons instead of following the next hype is exactly why I'm using Scala.

Scala's strength is shipping efficient, reliable and fault-tolerant software at a rapid pace.

[1] https://www.youtube.com/watch?v=iPitDNUNyR0

Static typing is certainly a factor, too. Scala allows me to not only design APIs which make it hard to be abused or misused, it makes it possible to encode many things I care about into types so that “wrong” code won't even compile.

With macros, there is now a whole new breed of libraries which add support for types to tasks which were to get wrong before, for instance

- the whole type provider business where one points to soma data source (like a database) and tells the compiler to figure out the right types on it own

- compile-time checked and SQL-injection-safe string interpolation like sql"""SELECT * FROM PERSONS WHERE $predicate"""

- sqlτyped (github.com/jonifreeman/sqltyped) which can compute types from SQL strings

- macros which transpile Scala code to JavaScript, inline (jscala.org)

- macros which can synthesize typeclass instances on-the-fly, like used in Scala's new serialization library (speakerdeck.com/heathermiller/on-pickles-and-spores-improving-support-for-distributed-programming-in-scala)

- Scala's async library (github.com/scala/async)

Regarding collection literals ... it certainly isn't that important in languages like Clojure where performance is not of great importance, because everyone just picks the one which come with the language and hopes it won't be too bad. Implementing new collection types is just not common here (like in many other untyped languages like PHP, Ruby, Python, JavaScript, etc.).

In Scala, blessing a few chosen collection types with special rules and syntax just won't fly. Developers demand first-class support for all collection types including the ones they define themselves.

Reserving some special rights which no one except the language creators are able to use just gives them an unfair advantage. All implementations should compete on the same ground so that the best one can win, and not the one which benefits from special-cased, hard-coded syntax rules.