Hacker News new | past | comments | ask | show | jobs | submit login
Metascala – A JVM written in Scala (github.com/lihaoyi)
139 points by ot on May 25, 2015 | hide | past | favorite | 53 comments



Never thought I'd see this up here. I wrote this, if anyone wants to ask anything about it


I've seen a few of the really impressive projects you've done over the past several years. How do you manage to stay so productive in your spare time?


Not related to the project, but have you got any suggestions on good materials for Scala learning?

Materials I've read / watched / done so far:

  - Martin Odersky's Functional Programming Coursera course
  - `Scala in Action`
  - 47 Degrees' Scala Koans
I'm currently reading `Scala in Depth`.


I read through Scala for the Impatient when I was initially learning Scala a few years ago. It's relatively new and written for version 2.10. Scala is on 2.11, but compared to 2.09 → 2.10, the changes are minimal. The book itself cuts out a lot of the BS and gets down to what a user wants to know about the language (features, syntax, how it differs from Java, etc).

Most Scala books were written pre 2.10, which makes them outdated enough that it would be hard for many new adopters of the language to get much out of them.

First 1/3 of the book is free online. The entire book is only $20 or so on Amazon.

http://www.horstmann.com/scala/index.html



With so many features, and so many of them built mainly for library developers, learning the language by yourself can be very daunting. I wonder where I'd be if I had not done a whole lot of the learning from reading a lot of idiomatic code written by well known developers. It's way too easy to end up writing Java with functions, or falling into common pitfalls that make code very hard to maintain in the long run.

If you've done the first Odersky course in coursera, the one he does with Roland Kuhn and Erik Meier is a very natural progression that will be far more practical than the original.

There are good series of blogs out there that, although somewhat outdated at times, explain concepts far better than the IMO dreadful Typesafe documentation. The twitter school, for instance, will at the very least give you names for features that, if you saw in code, you'd not be able to easily search for, due to their very generic syntax: For instance, context bounds. Another good set of blog posts comes from Bill Venners in the Artima website.

Good luck with your quest!


Note that Twitter's Scala School is very out of date, though I'm told an updated version is on its way.


Functional Programming in Scala is amazing. Highly recommended even for non scala programmers.

Second, come check out the #scala channel on Freenode. It's fantastic, has a REPL bot so you can show everyone in your compiler issues and get realtime help, tons of people, always active, very beginner friendly. Give it a shot.


> very beginner friendly

Unless somebody like Tony Morris needs to feel better about his day. The average IRCer is really nice but (at least when I was over there) there is an ugly and abusive side that almost burned me out on Scala.


Lets not mince words; it's not "somebody like Tony Morris", it's just Tony Morris.

But yeah, it sucks and I wish the community would do something. We don't need a big policy / code of conduct / whatever, we need an adult recognition that one individual is a problem.


He has partners in asshattery, but yeah, it's mostly him.


1. Skala Koans https://bitbucket.org/dmarsh/scalakoansexercises/wiki/Home

2. "Functional Programming in Scala" http://www.manning.com/bjarnason/

3. stackoverflow.com



Echoing any recommendations for 'Scala for the Impatient' - it is a fantastic beginner/intermediate book that gets you productive pretty quickly, so much so that I'm planning on having an intern who has no Scala experience go through it this summer.


Likewise, any tips for learning Scala without knowing/learning Java first?


Have you used any imperative, statically typed language that's compiled, such as C#, C++ or Swift? Previous knowledge of a purely functional language would be useful as well, but not required. Knowing Java helps with debugging and working with libraries built from Java, which depending on what one is doing (Android for example or using JSoup for scraping), is inescapable. You'll end up having to semi learn Java along the way, at least from what a friend told me (I already knew Java previously, so cannot comment on that). Scala, like C#, has extension methods to make dealing with Java libraries easier though, but it still doesn't solve things like JDK annoyances (type erasure).

If you're only familiar with scripting languages, it's going to be sort of a leap into learning Scala and could be done I think, though probably not the language I would recommend starting with since it's easy to abuse. The syntax is nowhere the same, but Scala sort of reminds me of the "Perl of statically compiled languages" in that you can write some really obtuse code without even trying if you're not careful. I don't think that's a bad thing necessarily, because it's a language that gives you a lot of flexibility (and more than enough rope to hang yourself). If you work with someone else, be sure to discuss some sort of agreed upon standard to using Scala, else you'll end up with some messy code.

Scala looks a bit like Python in parts as well as Swift, but I consider it more closer to a progressive version of C# with F# mixed into it.


Unfortunate given that my advice is sort of based on Java...

When I was learning Scala, I was intimidated by a lot of the syntactic sugar that it had. It's a calming effect to realize that all of that desugars into very simple, regular things. Java happens to be a very regular, unsurprising language.

So despite how surprising/special Scala may seem, under the hood it's just a bunch of methods.

There will be an acclimation period where you eschew the fancy stuff and stick with the Java style. But as you encounter the same structures and pain points, you will begin to see which Scala structures you like and will become comfortable with.

Find a style guide. Don't go crazy with point free. In general, it's a bad idea. (It's good if your DSL is super sexy/robust/easy to understand)


Advice needed. If I don't have Java or .net background would you recommend f# or Scala? I heard Scala is much more complicated than f#. I only know scripting languages like python and ruby.


Do you prefer to jump in at the deep end or to ramp up gently? F# is a more elegant language that will force you to go straight into functional style; Scala is a bit of a hodgepodge but you'll be able to write code that's very similar to Python, at least to start with.


Scala can be pretty similar to Python, if you choose to program in that style. You can then gradually expand into using more features. It's the path of least resistance, if nothing else.


F# is a better starting point, giving its ML syntax that you can easily transport to OCaml, Haskell, Swift.

I also like the language is functional first, meaning you are not forced to place functions inside companion objects or classes.


Swift's syntax looks way more similar to Scala than to OCaml or Haskell.


Between this, Scalatags, and the rest of the stuff you do, you stay impressively busy! I'm not sure what this would be useful for, but seems pretty awesome :)


Did you thought about integrating it with JikesRVM or Graal?


I personally think this is super cool. Keep up the good work and I can't wait to see more.

Thanks!


So now, your Scala can be both slow to compile and also slow to run!

Seriously though, that's a pretty cool project.


Writing something in scala feels like a waste to me. If they were to narrow the grammar of the language down, compiler time would decrease and the code rot situation would be much better.

Instead, teams that want to grow big in scala have to come up with their own agreements on which features not to use.

The deeper concern I see is the lack of introspection and defensiveness in the community when the critique is brought up. I've never seen programmers before take such offense and have to resort to banning programmers criticizing feature bloat.

It's not a personal attack or trying to kill Scala. There's a legitimate feeling the language tacked on way too much, too soon.

When it comes to a programming language, negative comments won't kill it. All languages grow their warts over time. Look at all the critique C absorbed.


The grammar is fine, probably simpler than Java's (e.g. try/catch is a much less specialized construct, type inference is applied much more consistently). What specifically would you look to cut?

Scala does seem to attract a disproportionate amount of criticism. And a lot of it is vague, nonconstructive, or simply wrong (and "writing something in scala feels like a waste to me" is nonconstructive in my book). Some defensiveness is warranted.


The grammar isn't a problem (at lest not for the compiler). Scalac is pretty fast at giving you error messages. It's when your code is legal that you really have to wait. I'd speculate that it's the code generator that's the problem.


Grammar ends up becoming a problem on a few fronts. I highly recommend Principles of Compiler Design [1].

One safe rule of thumb is, as grammar and rules expand, the performance and simplicity of the compiler begin to deteriorate, particularly if features aren't added carefully.

The second is in practice, the additional grammar gives you far more ways to tackle the same problem. The is the conundrum that metaprogramming gives in languages like Ruby. Teams begin the phase of coding to the domain before letting other things fall into place.

Thinking ahead is good, but I think the issue is Scala's grammatical flexibility makes it too tempting to dig ourselves into a DSL before we're really ready. This is why we'd get so grumpy when PM's came to us asking for new features that'd require a refactor. We bought in to our own idioms, if we programmed things without the DSL, we'd be more nimble.

And we're not dumbasses. We're experienced programmers with experience programming in that domain before, but the target always ends up moving for reason's outside of engineering's control. From experience we know, DSL has no place until you're very secure in what the needs of the business is.

That said, no codebase is future proof to the rest of the team needing last minute changes - but I feel scala forces you into design decisions too early, and we thought that was an advantage.

Actually, I liked scala for not having Java's boilerplate. Then I began to miss those old things java forced you into.

It was painful, because the ego investment the team had in scala was huge. A lot was on the line. Crow had to be eaten.

[1] http://en.wikipedia.org/wiki/Principles_of_Compiler_Design


It's worth pointing out that it's a case of YMMV. There are clearly those who haven't had success adopting Scala. I've seen Scala used successfully by multiple teams, but I suspect a strong culture of code reviews may have been a significant factor in our successful adoption of Scala. I would be disappointed if they tried to remove features from Scala, the extra features really do come in useful when used selectively.


If a good team nails the guidelines down before hand on which idioms they plan to use and stick to them, yeah.

That is a rare qualifier to meet in my experience, many teams will be ecstatic about scala. Many times we'd be hiring java programmers who felt that scala would make them more relevant in the startup world.

Next is the situation where it would be, "Just hold keep holding on..." until a. the language finally clicks b. the whole system is in scala. c. Both.

Scala did just click. And there never was a best solution to solve a problem. Python programmers were canned, we had the floor - those "distractions" were eliminated. We just got in this vicious cycle while we burned away our runway.

Now we look back, bitter, and make excuses. What hurts my pride to say is, Why didn't we just pick the safe bet - the Python or the Java.

I upvoted you. If you got a smart team and can make scala work somehow, all the power to you. I think you should make a story about how you scaled your code so the rest of us can learn


> If a good team nails the guidelines down before hand on which idioms they plan to use and stick to them, yeah.

So, like we used to do with C++? Anyone old enough to remember "C++ is fine as long as you define a well specified subset and stay away from certain features"? Even today, this is still what Google does about C++.

Hopefully we learned that this is never a good sign about a language.


In our case it wasn't about enforcing the use of a subset of the language, but rather being selective about when to use more advanced features. Having the advanced features of Scala available has proven to be very useful for us, but we don't go to town looking for a place to use every feature we can at every opportunity. It's a lot like chrome on a car - selectively and tastefully used it looks good, but making the car doors and roof out of chrome isn't a good idea.


I think this confirms the importance of a language restricting your choices, not only for type safety, but also for architectural soundness. Actually, it sounds like your team needed Go.


Yeah, you're given enough architectural rope to hang yourself with. That said, why make a DSL? It's very very rarely a good idea, says I, an armchair architect. I use Scala like Java with pattern matching, immutables and map/fold/filter. Works well so far.


I've found that DSLs which are well supported by theory tend to nearly always be a good idea. For example, Spire's mathematical data structures (group, ring, field, boolean algebra) tend to be useful, as are Scalaz's monad/monoid/applicative.

One which I haven't seen out there in the world, but turned out to be a fantastically useful, is the Free Boolean algebra. It's a functor FreeBool[_] which is also a boolean algebra. It's Free because it has the property that for any function f: T => U, U a boolean algebra, there is a natural transformation nat(f): FreeBool[T] => U which is also a homomorphism (nat(f)(x & y) = nat(f)(x) & nat(f)(y), etc).

Even though things were not actually pinned down (and still aren't), I'm confident this is a good choice - thousands of mathematicians are very rarely wrong.


Most of the time is spent in the typechecker.


> There's a legitimate feeling the language tacked on way too much, too soon.

A lot of people certainly feel this way, and this feeling has given rise to the first crop of post-Scala languages: Ceylon and Kotlin.

I'm sure there will be more.


People are still asking for examples, not for some popularity vote on feelings.

(Also, the designer of one of the languages mentioned was completely unaware of Scala when he created his own.)


Don't think that's the case with Ceylon or Kotlin, perhaps with some other "post-Scala" JVM language such as Gosu or Fantom, but definitely with Groovy. It's creator James Strachan wrote in http://macstrac.blogspot.com/2009/04/scala-as-long-term-repl... "I can honestly say if someone had shown me the Programming in Scala book by by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I'd probably have never created Groovy".


No, you are confusing people.

The person is James Strachan, he created Groovy. Since then, he abandoned Scala and he's now a committer on Kotlin (or was, not sure if he still is, he tends to bounce around projects a lot).


No, I'm right. I didn't think of James Strachan. (He by the way stopped contributing to Kotlin around October 2012. You're welcome.)


Ok, so who is this person you are referring to?


So what _is_ your criticism?

The grammar of the language has no relationship with compilation speed.

What did they "tack on way too much, too soon"?


println(println("inception"))


[flagged]


You're kind of missing the point, TheShrek. The point is to prove it can be done.


Everything mostly can be done... The REAL point is : Should it be done. IMHO : No! So I don't think I'm missing the point.


Rewind to 2006. Should we implement Python in Python? Of course not; it's not useful! PyPy should not be done! It's literally 50 times slower than CPython!

Fortunately, the PyPy folks have stuck with it, and PyPy is now faster than CPython across the board. Many other languages have also been implemented using its considerable infrastructure. Compiler experts have been minted from scratch. Should it have been done? Yes.

Even with the vulgar assumption that "practical utility" is the only source of good, this type of project is good.


"Metascala is barely 3000 lines of Scala, and is complete enough that it is able to interpret itself metacircularly. [...]

The goal of Metascala is to create a platform to experiment with the JVM: a 3000 line JVM written in Scala is probably much more approachable than the 1,000,000 lines of C/C++ which make up HotSpot, the standard implementation, and more amenable to implementing fun features like continuations, isolates or value classes."

That seems pretty neat to me.


Any reason why it SHOULDN'T be done? I fail to see any downsides to its existence.

"Why not?" is a perfectly good response to "why?", especially on a site called hacker news.


The point, from the project page:

> The goal of Metascala is to create a platform to experiment with the JVM: a 3000 line JVM written in Scala is probably much more approachable than the 1,000,000 lines of C/C++ which make up HotSpot, the standard implementation, and more amenable to implementing fun features like continuations, isolates or value classes.


My take on this: Experimentation is a form of learning. Whilst the end product may seem trivial, there may be meaningful discoveries during the journey.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: