If you have to use the JVM and don't want to code in Java, your options are Scala or Clojure (maybe Groovy?). Then it comes down to essentially the tradeoff between a static typesystem vs macros (imho, ymmv etc).
I second the reccomendation for Chuisano and Bjarnasaon's "Functional Programming in Scala".
Kotlin and JRuby appear to be your next best options after Scala and Clojure. Both have growing user communities (thanks to Rails in the case of JRuby and Android in the case of Kotlin). Both solve the library problem by offering seamless compatibility — with Java and MRI (with some caveats [1]) respectively. If you start a long-term project in JRuby you'll be able to hire Ruby programmers down the line. Kotlin is a less safe bet in terms of hiring but I except its prospects to become clear soon, probably within a year.
Apache Groovy is really for those quick'n'dirty scripts manipulating and testing Java classes -- that's its original use case. Later it added a MOP so Grails, a knock-off of Rails, would work, though Grails use is fast waning -- not many people upgraded their websites to Grails 3, or start new projects in it. Gradle then started using it as a DSL, so there's now a few thousand sites using the non-Turing Complete features of Groovy in 20-line build files. When Groovy 2 came along with static compilation and eventually targeting Android, not many people upgraded, prefering to stick with Groovy 1.8. The programmer who put the static compilation into Groovy now works at something else, and no-one's replaced him. So really Groovy's not a contender for building systems in the way Scala or Kotlin are, which were both designed for static compilation from the ground up. Nor does Groovy have syntactic macros the way Clojure has -- you need to write screes of Java code to implement a single annotation.
What about Kotlin? Its 1.0.0 has been recently released and comes as "pragmatic" Java, bringing a lot of useful features and having more concise yet readable syntax. Also, the Java interop is really good (better than that of Scala, for instance).
The Java interop is worse than that of Scala - Kotlin can't represent Java's variance (i.e. existential types) so it has hacks to avoid it. It doesn't have the maturity or ecosystem of Scala. The "pragmatic" design feels like they took a grab-bag of features from Scala and implemented all the simpler use cases individually, with no appreciation for the underlying coherence; IMO the language will not be able to evolve because it has too many special cases. And Ceylon does everything it does but better. Kotlin is a triumph of marketing hype, nothing more.
What do you mean by inability to represent existential types? Kotlin has in/out/star projections, which are be used for corresponding Java variance. A note about Scala interop: forget the existential types, in Scala I cannot simply use interfaces with bounded type parameters without some dirty hacks, just because compiler will generate weird errors and/or incorrect bytecode. I am sorry, but this makes it just impossible to work with Java in Scala.
The whole point of Kotlin is that it does not need its own ecosystem - it is intended to be used seamlessly (unlike Scala or Ceylon) and, like, today with all existing Java libraries and frameworks. Sure, more idiomatic solutions would always be nicer to work with (and they would probably appear at some time), but you do not need to wait for them - just go with your existing codebase.
As for the tooling, Kotlin IDE support way better than Scala's. It is not surprising, however, due to the fact that main language kontributors are JetBrains.
> What do you mean by inability to represent existential types? Kotlin has in/out/star projections, which are be used for corresponding Java variance
Hmm ok, that does seem to provide equivalent functionality. Is it implemented?
> in Scala I cannot simply use interfaces with bounded type parameters without some dirty hacks, just because compiler will generate weird errors and/or incorrect bytecode
Example? I've never seen that happen and I've done a fair bit of mixed Java/Scala.
> The whole point of Kotlin is that it does not need its own ecosystem - it is intended to be used seamlessly (unlike Scala or Ceylon) and, like, today with all existing Java libraries and frameworks.
Scala was intended for that at first. It turned out to be a bad approach. Interfaces are important, library APIs have a surprisingly far-reaching impact on the code that calls them.
> As for the tooling, Kotlin IDE support way better than Scala's. It is not surprising, however, due to the fact that main language kontributors are JetBrains.
That one specific tool perhaps. In terms of the much larger ecosystem of third-party JVM tools, Scala support is naturally more mature.
With more than 2,000 people in its Slack channel alone, I think Kotlin is a bit more than marketing hype (to compare, Java has about 500 and Scala 450).
As for your other point, tying Java interop with existential types is pretty absurd.
I'd say both Kotlin and Scala are about equally good when it comes to interoperating with Java.
> With more than 2,000 people in its Slack channel alone, I think Kotlin is a bit more than marketing hype (to compare, Java has about 500 and Scala 450).
So do you really think Kotlin is used 4x as much as Java? Or is it possible that this is a pretty poor measure of actual usage?
> As for your other point, tying Java interop with existential types is pretty absurd.
Only as absurd as Java. Java libraries contain methods that return existential types (e.g. List<?>) everywhere. If you can't represent that type in your type system (note that it's not the same type as List<Object>), you can't have full Java interop.
I don't know what libraries you are looking at but these are extremely rare these days. At least for the popular ones. You seem to be talking about Java like it was written eight years ago.
I gave List<?> as a simple example; usually we're talking about List<? extends Foo> or similar. Anything that involves a ? anywhere is an existential type, and they're the only way to write a whole lot of things safely, e.g. any kind of callback has to involve a "? super" type to avoid disallowing perfectly valid code. Unless the Java community has abandoned generics and decided to go back to using casts everywhere there's really no alternative in the language.
Exactly, I love working with Clojure, never disappoints, CSP is amazing and even though I understand the benefits of type safety too many people get hang up on that and write type safe incorrect code that does not do well in production. Kotlin seems a bit less complex than Scala, I might give a shot to that. Thanks for the book recommendation though!
I second the reccomendation for Chuisano and Bjarnasaon's "Functional Programming in Scala".