FYI, Simon Marlow is incredibly big in the Haskell community and is the leading expert in Multi core parallelism via STMs in Haskell. Some of the work he has done for the GHC compiler is frankly fantastic.
This is incredibly big news in the Haskell world. The FP community is starting to break boundaries with the industry in the last 5 years with Clojure (and Scala's functional support).
Currently, FB seems to use haskell only for basic lexing and parsing[1]. It would be tremendous to see them using it at FB scale. I think there's going to be gain for both parties - For FB, the publicity and hacker outreach, and Haskell - industrial reliability.
> The FP community is starting to break boundaries with the industry in the last 5 years with Clojure (and Scala's functional support).
There is very little evidence of that. Do a search for Scala, Clojure or Haskell on the main indicator web sites (job boards, StackOverflow, TIOBE, indeed.com, etc...) and you'll find they are as insignificant today as they were five years ago compared to the mainstream languages.
While this comes up every single time somebody talks about functional programming making inroads, I still consider it a fairly unhelpful indicator. First, nobody's arguing that big companies are suddenly doing the bulk of their development in these languages. It's about using them at all. Second, job postings may be lagging and distorted in their own way (e.g., junior positions are much more likely to be publicly advertised, and so on).
Anecdotally (n=1, of course) I have seen and heard tremendous resistance to clisp in the industry I work in 5 years ago. Now, clojure and Scala are being adopted and recommended by even conservative PMs and architects.
The choices are way more today (Almost a tautology given old choices don't really vanish into thin air) and on more reliable platforms with tried and tested libraries.
anyway the spirit with which i meant that statement was that it is far easier to convince "senior management" and others with power to let stuff be done in clojure / Scala than it was to convince them about ml or clisp five years ago.
And I am not even talking about "ivory tower Haskell" (which i love using).
I think when people talk about functional programming being used more in industry, they don't imagine it replacing Java or Python as the lowest common denominator language everybody uses. Rather, they imagine either small companies or small teams within larger companies using functional programming to be more productive.
So the sheer number of people using functional programming may not be very high even if functional programming is making inroads into industry. Now, I don't know whether functional programming actually is making inroads into industry or not, but my impression is that anybody relatively good at it could find a very nice functional programming job fairly easily.
Not to mention that they are also referring to functional programming being used within those existing languages. Even Visual Basic contains a pretty capable functional subsystem these days, with lambdas, higher order functions and so on (albeit lacking concepts like immutability and referential transparency).
Job boards have one weakness: They reflect supply and demand. There is not much demand for Scala programmers. This may be because all positions are filled. Or, a more probable cause, that Scala programmers are not hired on Job boards.
Note that I have had 5-6 Jobs. I do FP for a living. And I have never been hired through a Job board advert. I think this is common with many FP people. Usually, when you want to hire FP people, you go to the mailing lists and community and get the guys you need. It is much more effective, but it won't show up in the Job board search at all.
> Whenever somebody quotes growth numbers instead of absolute numbers, they are trying to hide something.
No, it is the data you asked us to look at and it refutes your claim that "they are as insignificant today as they were five years ago". They are _less_ insignificant. They may still be several orders of magnitude lower, but your claim is still incorrect.
Just take absolute numbers: According to indeed, there are over 1000 open jobs mentioning Scala in the US, and over 750 in the UK. Taken together, the number of open Scala jobs in both countries exceeds the number of open Groovy jobs. It's still small compared to Java jobs (to be precise 1.5% in the US, close to 3% in the UK), but Java is by far the most demanded language, so this is not surprising. To say there is no pick-up in adoption of functional languages is disingenuous, IMO.
And most of those job ads mentioning "Groovy" are for using groovy as an add-on to Java, while those mentioning "Scala" are usually for using Scala as the primary language.
And, I should add, given this level of demand it's a good thing we just finished the Scala massive open online class with more than 10'000 developers completing the course with success.
> This is incredibly big news in the Haskell world.
I'm puzzled why you think this is good news for Haskell. The lead developer and architect of GHC and authority in parallel Haskell is announcing that he's taking a position that won't allow him to spend much time working on GHC, and he might even stop using Haskell completely.
It's great news for Simon but terrible news for Haskell.
Pedant calling - Erlang is a functional programming language that has been doing very nicely thank you in a lot of big companies (Amazon, Facebook amongst others) for a good while now.
Other Functional languages are being used more as well, but since a huge amount of financial software sits on either a .Net or a Java stack, using FP languages which can easily interop with the rest of your libraries makes life a lot easier.
That's more true in certain Enterprisey, conservative parts of the financial world, backoffice bank records and that kind of thing. In investment/quant/trading parts there's more experimentation with other languages, both functional languages like F#, and even particularly "weird" languages like K (an APL successor).
While k is extremely terse, a lot of investment banks these days are using systems which utilise q and it's associated kdb+ database. q is basically a wrapper around k which makes it a lot easier to read and code in and which allows for sql-esque select statements from databases. This combination makes it far more flexible and useful for dealing with financial databases than straight k.
My rule of thumb these days is that the smaller and more obscure the hedge fund, the more likely it is that they're using an unusual set of languages. The more PhDs they have, the more likely they're using something weird like K!
I used K (well, actually the combination of Q and kdb+) in my previous job at an investment bank, and I've used it (to a lesser degree) in my current job at a hedge fund (disclaimer: I have a PhD...).
K/Q is a beautiful language - it's uncompromisingly terse, extremely powerful, and very fast. The interop with kbd+ (a lightning fast, column-oriented, in-memory database) is gorgeous.
It has an unfortunate reputation for obscurity, stemming mainly from the people who think it's fun to write your entire program as a K one-liner and remove all the whitespace. You can write perfectly clear code in Q - super modular, concise, and well documented. It's just that a lot of Q programmers don't bother.
Hardly. They used to be in love with Smalltalk and even Objective C, even before it was cool with Apple and iOS. Also, lots of LISP. The financial world is incredibly hungry for talent and whatever technology can give them an edge over the competition.
The financial world is a big place. There's everything from backend systems at banks to stock exchanges, trading houses, stockbrokers, all with different requirements. I think perhaps the most ancient systems are the backend money ones at banks since there's not much money in scraping speed out of them, but I'd love to hear from someone in the field?
Certainly the enterprise stuff at Spanish banks is run on Cobol, and there's still a fair amount of active Cobol development going on. (By enterprise I mean the bread and butter stuff, accounts etc.)
No idea about the trading / derivatives / whatnot, though.
The core financials may be in COBOL and assembler, but these days most things talking to them will be Java or C#; the middleware will most likely be Java as will many of the front-end apps.
And plenty of banks are migrating away from COBOL on the core systems (slowly), and for them Java is usually the new COBOL, and the JVM is the new s390 microcode.
There is almost certainly a hell of a lot of Fortran and COBOL floating around as well. Most of it is likely at retail banks where things centre around end-of-day processing runs and batch processing.
There's this perception that Scala is less FP than a language like Clojure and I don't understand why, when in my (albeit limited) experience, I found the contrary to be true. This impression that's perpetuated is IMHO only folklore coming from people that only played around with these languages superficially.
Scala feels more like a functional programming language to me because:
(-) Code written in Scala tends to be more FP. One of the popular Scala frameworks, Play 2.0, is using Iteratees for dealing with responses and I/O in general, a concept used in Haskell, being pure awesomeness if you want to do Comet/Websockets. Play itself feels more reactive than Node.js, even though Node.js was built from the ground up for this. Also monadic types are used everywhere, from the standard collections to the Maybe monad (Option) to Futures/Promises. In Clojure you rarely see this, because Clojure is so dynamic that working with complicated container types is unfeasible.
(-) the Scala collections library provides more variants of immutable data-structures optimized for different things and those immutable data-structures in Scala are doing the right thing - first they play well with each other (e.g. mapping over a Map produces another Map if the function returns a tuple, but produces a Seq in case it returns a single element and there are many more examples where Scala simply knows what you mean).
Many times you pick concrete implementations like Vectors because of the provided invariants (e.g. indexing in O(1) or efficient adding/removing of elements on the right) and changing those invariants is IMHO bad design. Doing operations on a Vector like adding elements shouldn't produce a list, even if that list is lazy or whatever other reason. In Scala you know what operations are going to behave lazily because such invariants are encoded in the type. Adding/removing/transforming elements from a lazy Stream or view or other kind of lazy collection is still going to produce a lazy collection. And this is extremely useful in languages where laziness is not the default.
From the perspective of its rich collection of immutable collections, Scala is more functional than popular ML languages in general, except maybe Haskell. In Ocaml idiomatic code includes the usage of mutable collections, while in Scala the mutable versions are second-class citizens - and before you say that Clojure doesn't have mutable collections, think again - arrays are part of standard code and working with Java collections is quite comfortable and given as example in Clojure books.
(-) STM and atoms are great. I use STM in Scala a lot lately. However stuff like atoms and STM are still mutable state and come with many of the same gotchas as plain vars and pretending like this is otherwise is dangerous. Precisely for this reason it's useful to be able to implement your own atoms and your own STM mechanism that is more suitable to the problem at hand, that better encapsulates state and so on and because of the reality we live in, the internals of such components can get messy, while still keeping referential transparency for the outside world or other kinds of invariants - STM in Scala is a library (and quite nice in fact since it forces you to use Refs inside transactions with compile-time checks); Erlang-style Actors are a library too; Futures/Promises are a library too. All of them created with Scala.
(-) Scala is OOP while Clojure pretends like it isn't. I'm using "pretends", as multi-methods were introduced in CLOS, an OOP extension to Common Lisp, which made no effort to hide the fact that multi-methods are still all about OOP.
But just because OOP is very efficient (and has been used) to hide mutable state, that doesn't make OOP any less orthogonal to functional programming. Maybe tying up data with functions operating on that data leads to bad design, but besides the fact that type-classes are also part of idiomatic Scala and used everywhere, OOP is actually a quite nice and comfortable ad-hoc mechanism for polymorphism, being quite orthogonal to FP. Clojure is only able to have multi-methods and protocols because it is dynamic, but as far as FP is concerned this has drawbacks. Also in regards to multi-methods, I've seen code in the wild that dispatches on some global state instead of their arguments, since you can dispatch on anything and I'm really talking about mutable state.
IMHO I can't find evidence that Clojure is more FP than Scala. I'm more experienced with Scala than Clojure, so maybe this is why. But IMHO, Scala is more potent if you want to explore concepts of functional programming, being in my opinion the second choice after Haskell.
And I do think that this perception only has legs because Scala has a Java-like "while" loop and doesn't hide the fact that "vars" are really vars.
Your macro argument is a terrible one, you can do exactly the same thing with any sane language (and indeed I have done, getting python to create boilerplate java)
You dont need macros for this (which your post shows since I cant see any defmacro used there)
I think idiomatic Scala code should be FP. And although you can just quite Java-esque code in a slightly more concise way in Scala; I do think that people are starting to see that Scala is more than that - it introduces new ideas and idioms.
Probably because many university students are taught Clean (the 'other' purely functional lazy programming language) [1] which is developed in the Netherlands.
I would not bet on that. Utrecht (http://www.cs.uu.nl/wiki/UHC) is in the Netherlands, too. I also expect/guess that it has more CS students than Nijmegen
So every dutch university teaches a language they themselves wrote the compiler for. The university of twente used to teach FP in a miranda dialect called amanda, which ofcourse was developed at utwente :p
Yeah, Utrecht University has some of the world's leading Haskell researchers, I think it is taught in Groningen as well. In Nijmegen they teach Clean, which is very similar to Haskell.
This is incredibly big news in the Haskell world. The FP community is starting to break boundaries with the industry in the last 5 years with Clojure (and Scala's functional support).
Currently, FB seems to use haskell only for basic lexing and parsing[1]. It would be tremendous to see them using it at FB scale. I think there's going to be gain for both parties - For FB, the publicity and hacker outreach, and Haskell - industrial reliability.
[1]: http://www.haskell.org/haskellwiki/Haskell_in_industry