I have a good analogy to explain this. Take this OCaml program.

    let sum_file filename =
      let file = In_channel.create filename in
      let numbers = List.map ~f:Int.of_string (In_channel.input_lines file) in
      let sum = List.fold ~init:0 ~f:(+) numbers in
      In_channel.close file;
      sum
    ;;

    def sum_file(filename):
      with open(filename) as f:
        return sum([int(x) for x in f])
       

OCaml is based around ML-style typed records, which are exactly what you need for manipulating languages.

I am not a type safety guy, but when you write languages, there are tons of nested conditionals. In Python in JS or C, you end up with bugs in those corners. It is quite easy to crash any non-trivial parser, like the CPython parser, or Clang's parsers, etc. The type safety that ML offers helps a lot with this.

The code for writing parsers and interpreters is just SHORTER in OCaml than in Python. I used to think Python was the most concise language. But no, it depends on the problem domain. Try it and you'll see.

Someone saying the same thing here:

http://flint.cs.yale.edu/cs421/case-for-ml.html

    sumFile :: FilePath -> IO Int
    sumFile file = sum . map read . lines <$> readFile file

That hasn't been true for a few years, but it was true for quite a while. At this point Pugs is fairly out of date and (I believe) abandoned, as it's main developer discontinued development. For a long while it was the most advanced Perl 6 compiler though, and from what I understand a lot of it's rapid advancement was attributed to it being written in Haskell (Perl 6 and Haskell share a good portion of advanced features, so that may have helped).

The current state of support of the various Perl 6 compilers can be seen here[1].

1: http://perl6.org/compilers/features

Do you know how this would look in Haskell?

    # Return K most common lines in a file

    def top_k(f, k):
      counts = collections.defaultdict(int)
      for line in f:
        counts[line] += 1

      return sorted(counts.items(), key=lambda x: x[1], reverse=True))[:k]
    
       

    top xs = map fst $ sortBy (compare `on` Down . snd) $
        M.toList $ foldr (\x -> M.insertWith (+) x 1) mempty xs

I don't begrudge anyone if they want to use Haskell or OCaml for everything.

But personally I am interested in using ML-like languages for language engineering and not general purpose programming (maybe Rust could change that, but I'm not convinced). I program in 5+ different languages regularly so I've dealt with all the friction involved, mainly by explicitly designing systems as heterogeneous collections of Unix processes.

* In ocaml you need to pass some "comparator" parameters around if you want top_k to be polymorphic. Its similar to haskell type classes but you need to be explicit... * Instantiating a polymorphic hash table is a bit ugly but thats more about the standard library... * There is no "default dict" function so you would need to initialize the counts yourself * Looping the file is not hard. Most data structures will have some sort of "fold" or "iter" function you can use. * The sorting function doesn't take keys, so you need to use a comparator instead. Its not that bad though - the function to convert a hash table into a list already returns a list of tuples.

It's conceptually similar, but just plain uglier in practice. A lot of day to day programming is just FILLED with stuff that is very elegant in Python, and not so much in OCaml.

Language engineering is something is very elegant in OCaml.

In my opinion, EVERY language is a DSL. I stopped looking for the ultimate language -- it doesn't exist.

While you might not see this dynamism as a big problem in Python, its something that I always found confusing about Ruby. Over there, people everything is a method call and people are super happy to monkeypatch extra methods to classes, which makes it very hard for me to understand the program by reading it. At some point, things get so dynamic that static analysis in your head gets too hard and you are better off just running the program and seeing if it worked...

In any case, I can't give you a concrete example right now because my laptop doesn't have ocaml installed on it but in my other post I already listed some of the bigger differences that the ocaml version has. The tldr is that some of the features you are using in Python aren't available in the ocaml stdlib (so you would need to write the sorting comparator the long way or create the helper function yourself) and that creating the polymorphic hash table is not very beginner friendly (this is a cost we pay for stronger typing). What I wanted to say is that the big things about the Python function (hash tables and generic sorting) are both things you can do in Ocaml just fine.

I'm curious, how did you think it would look in haskell? Can you give a code example of what was in your head?

I was saying that OCaml was awkward for this problem, and the reply was that OCaml is but Haskell isn't. I'm not convinced after seeing the Haskell code. Python is both semantically and syntactically a better fit for that problem.

Then what were you impling by saying:

> Do you know how this would look in Haskell?

Because that sentence seems to imply it would be very convoluted in Haskell. How can you imply it would look convoluted in Haskell if as you said:

> I don't know Haskell, so there wasn't anything in my head.

Anyway, onwards...

> I'm not convinced after seeing the Haskell code. Python is both semantically and syntactically a better fit for that problem.

You aren't convinced of what after seeing the Haskell code?

How can you come to the conclusion that Python is both semantically and syntactically a better fit for that problem if you don't know both Python and Haskell?

Can you give me the criteria you are using to come to that conclusion?

Here is a (more or less) identical implementation in Rust:

    fn top_k(f: Lines, k: usize) -> Vec<(&str, usize)> {
        let mut counts = HashMap::new();
        for line in f {
            *counts.entry(line).get().unwrap_or_else( |v| v.insert(0) ) += 1;
        }
    
        let mut counts = Vec::from_iter(counts);
        counts.sort_by( |x, y| y.1.cmp(&x.1) );
        counts.truncate(k);
        counts
    }

* Curly brace syntax. Not a property of ML languages--Rust and Swift are pretty much alone in the ML family in having curly brace syntax.

* Having to write & explicitly. Not a property of ML languages at all--it's a C++ism. In the type signature, the & is because Rust has first-class references--it deliberately does not abstract over them, unlike nearly every other ML (and most other languages, for that matter). The second case is a stylistic choice: Rust is (somewhat inconsistently) explicit about when you are taking a reference. There have been some proposals to remove this annotation burden.

* Rust doesn't have a find_or_insert method in the standard library (or a "HashMap with Default" a la defaultdict). Not a property of ML languages; in fact, Rust used to have this (see http://web.mit.edu/rust-lang_v0.11/doc/std/collections/hashm...). It was removed because the Entry API is seen as more general. It would be quite easy to add such methods or types back as sugar in a library (if someone hasn't already).

* By convention, Rust APIs that mutate things don't return the original. Not a property of ML languages, just a Rust thing (in Rust, you can get sugar for this a chain! macro, as demonstrated at http://www.reddit.com/r/rust/comments/2wssch/function_chaini...).

* Having to write out ".take(k).truncate" instead of just being able to use slice syntax like [:k] (in Rust, [..k]). This is not an ML thing; the reason you can't do this is because slices in Rust do not create copies, but references into the original vector. If Rust were garbage collected, this would be fine, but in Rust the counts vector would be deallocated at the end of the function, leaving the references dangling. We could clone it, but it wouldn't be any shorter and it would be wasteful compared to just calling truncate.

* Having to specify that the counts be represented as a `Vec` to turn the counts item list into a vector. This is not an ML thing; it is because in Rust, vectors are not privileged over other types of containers. Most MLs have plenty of syntactic sugar for lists; this is again something more C++ish.

* Using '::' instead of '.' as the path separator. Most MLs just use ., this is a C++ism. I believe it is related to Rust's desire to remain LL(1), but I can't remember the particular reason for this one.

* A big one that might seem like an MLism, but isn't: explicitly writing out the types. While MLs vary in how much type inference they actually support, most will infer them. Rust is somewhat unusual in requiring function-level type annotations. In particular, a language with global inference should be able to infer this signature for the function (given in Rust syntax):

    fn top_k<I: IntoIterator>(f: I, k: usize) -> Vec<(I::Item, usize)> where I::Item: Eq + Hash

* Having to write "let" and "let mut". I don't think there is any ML language that doesn't differentiate between variable creation and variable assignment. Similarly, I can't think of a ML-family language that doesn't differentiate between mutable and immutable bindings (however, there was a quite serious proposal by one of the core Rust developers to do this: http://smallcultfollowing.com/babysteps/blog/2014/05/13/focu..., and it could be made less boiler-platey for the common case, as Swift does, by having a separate var keyword, at the cost of losing power while pattern matching).

Thus, here is a hypothetical implementation in an MLish language that switches the non ML-ish Rust-isms to quite reasonable ML-ish alternatives (and has a garbage collector, presumably, but if you don't like that just pretend slice is a copying operation in this language):

    fn top_k(f, k):
        var counts = HashMap.new()
        for line in f:
            counts.find_or_insert(0) += 1

        from_iter(counts).sort_by( |x, y| y.1.cmp(x.1) )[..k]

  let sum_file filename =
    with_file filename 
      (fold_lines ~init:0 ~f:(fun a l -> a + int_of_string l))

See my other comment -- how does top K lines in OCaml look? I recall that was in the book too, but couldn't find it. I remember it being fantastically ugly.

  let top_k chan k =
    let incr_count m l = 
      let n = try Map.find_exn m l with Not_found -> 0 in
      Map.add m ~key:l ~data:(n + 1)
    in
    In_channel.input_lines chan
    |> List.fold ~init:String.Map.empty ~f:incr_count
    |> Map.to_alist
    |> List.sort ~cmp:(fun (_,a) (_,b) -> compare b a)
    |> List.sub ~pos:0 ~len:k

http://www.fssnip.net/lf

Hammers in nails with an orange, and this folks, is how Black and Decker makes better drills than Makita.

The single reason that OCaml excels at compilers is due to pattern matching. It _removes_ all those explicit conditionals by making them implicit.

http://andrej.com/plzoo/

Python has excellent affordances for writing compilers, they just have to be used.

https://github.com/mrocklin/multipledispatch/

https://github.com/Suor/patterns

Do those Python libraries offer type safety? I don't think there is any way they can. If not, you might as well use OCaml.

As I said, Python is my favorite language, but there is no reason to write OCaml in Python or even shell scripts in Python. Why not use the real thing? The Python code still won't be shorter than OCaml, even if you use these fancy libraries.

At the same time, that has to be balanced with the ecosystem value of implementing your language in a language lots of others knows.

It's a worthwhile investment to implement your language with 4x the code in Python/Java/JS/etc. if you get more than 4x contributions because of it.

Maybe, but I'd say it depends on the quality of the contributions.

IMO it's actually advantageous to have a compact description of a lexer, parser, and AST that one person edits or very few people edit. That's your language design.

At one point I thought: "If ML is so great, then why don't you see it more often in the real world?" And then the revelation was that Python essentially uses ML to describes its AST:

http://svn.python.org/projects/python/tags/r32b1/Parser/Pyth...

This is Zephyr ASDL, an ML-inspired DSL for describing ASTs. A lot of ML people may recognize Andrew Appel from the authors list:

https://scholar.google.com/scholar?cluster=11682730813888505...

So my favorite language's AST is described with ML!!! And has been for 10+ years. (Python has a Grammar file as well in a different syntax).

I'm actually interested in a hybrid architecture: an interpreter where OCaml generates the byte code, and then C++ executes it. I think you can produce extremely compact and flexible interpreters with this architecture, and there are several other advantages to dividing it this way.

The downside is perhaps a more complex build process, but the OCaml toolchain is quite nice actually, and I have compiled it from scratch. It's head and shoulders above Haskell in that regard. OCaml can produce .o files and link with C/C++, so you still get one executable.

I looked at your Wren language which I like a lot. I did wish the front end was more high level and not in C, but that's just me :)

There are a lot of people coming around to OCaml. Facebook is using it for Flow and pfff language manipulation:

https://github.com/facebook/pfff (I suspect Google's code analysis tools would be cut down in size by 5x if written like this in OCaml.)

And Hack, the statically typed PHP, is written in OCaml. And there are several more minor languages like HaXe and another one that are OCaml.

So if you're looking for contributions from language experts, there's definitely a lot that are familiar with OCaml.

http://nathansuniversity.com/

And if you want to actually build a "real" programming language, I advise you to try llvm - it really takes away the pain of generating bytecode, and gives you everything you need to deal with the actual design of your language.

Possibly more digestable, I recommend the walkthrough of building "Egg" in the fantastic (and free!) Eloquent JavaScript by Marijn Haverbeke: http://eloquentjavascript.net/11_language.html

I get that it's a beginners tutorial and they have some constraints, but it starts off with "let's dream up a language" and then presents a super standard language, it's basically just Javascript with obligatory semicolons..

Any case, the intent was purely didactic, but I did implement a Scheme dialect based on half of that code. To be released some day...

Anyways, this is such a detailed series of tutorials I may be distracted for days bringing my half finished programming language back from the dead.