It's a what? Can someone ELI5 this for a lowly Python and JS dev - why would I want this?

----

I watched a talk about Scala's future [0] recently, in which the presenter compared the taglines of a few different languages (Go, Rust, Erlang, Scala), and how they relate to business pains or non-academic problems.

His conclusion about Scala's tagline - "Scala combines object-oriented and functional programming in one concise, high-level language" is that "no engineering manager in the history of software development ever thought to themselves 'hmmm, if only I had a language that fused OO and FP, I'd be able to solve my business problems'. That hasn't happened; that will never happen. This is an academic novelty that has zero relevance to any of us as professional software developers [...] this is not a business pain, this is an academic interest."

I felt a little bit like that when I read Egison's tagline: why should I care about this language?

[0] "The Last Hope for Scala's Infinity War" https://youtu.be/v8IQ-X2HkGE?t=15m8s

Nah, but I can ELI a person with some programming experience.

Pattern matching is super useful for writing code that spends a lot of time building and deconstructing syntactic data structures.

Non-linear pattern matching is useful when de-constructing large-ish expressions that might contain multiple occurances of the same sub-term.

Nonfree datatypes allow you to encode some constraints about your datatypes; e.g., "lists that are equal modulo permutation".

Supporting type-checked non-linear matching over nonfree datatypes allows you to encode a lot of constraints in the type system and syntax that would otherwise be encoded with lots of nested if-statements and only checked by tests, not by static type checking.

This combination of features could be very useful when implementing a lot of mathematical code that operates over algebraic structures. Which is why the language's homepage mentions the implementation of (features for) computer algebra systems as a sort of killer app.

> this is not a business pain, this is an academic interest

Not all innovation is motivated by the business pains of today's engineering managers. And often the solutions to those pains are not recognizable; e.g., thousands engineering manager in 1950 could've beneifted from CAD software but I bet very feew were sitting around saying "I could really use a good piece of CAD software right now".

Scala is another good example of that, despite a marketing complaints about messaging. No one asked for FP+OO, but Scala took off because the combination of FP and OO solved a lot of the pain caused by Java.

Substantially improving the ease of implementing new types of computer algebra systems could easily create a lot of business opportunities e.g. in providing improved analysis/synthesis tools for all sorts of CAD tools.

that said, I don't think this language hits the target. See the hodge operator example...

Egison appears to be a self-consciously academic language, which strikes me as a perfectly legitimate.

In particular, the goal seems to be to provide a more intuitive way of writing algorithms using extensible pattern matching. Now, I have worked on commercial software that has highly algorithmic parts (graph traversals, etc.) written in Erlang (a language that supports a powerful, but not extensible, form of pattern matching). I found the pattern matching facilities to be quite helpful in producing readable, concise, correct code.

The use case is narrow, but strikes me as real nonetheless.

It's a bit subtler than that. Scala is an industry programming language perfectly suited to business applications, but with a big academic and research input (remember that Odersky, its creator, was a big contributor to Java, an industry language) and with an initial emphasis on research. The mentioned talk is criticizing Scala's marketing blurb because it was written almost entirely from a research angle, and doesn't accurately and convincingly portray Scala's benefits for business. See the revised blurb at the end of the talk [1]. It's a question of being clever about marketing.

----

[1] revised blurb: "Scala is an open source programming language that helps you write correct code and modify code safely, while seamlessly reusing existing JVM libraries."

Egison features a customizable pattern-matching facility. Pattern-matching methods for each data type and pattern can be customizable by users. There might be a possibility that you thought syntax sugar is not syntax sugar.

This language is clearly an academic endeavor.

Disclaimer: I don't know what non-linear pattern matching on non-free data means either, but the fact that it's not explained to me is proof enough they aren't meaning to cater to non-academic users.

Egison makes programming dramatically simple!

    (define $twin-primes
      (match-all primes (list integer)
        [<join _ <cons $p <cons ,(+ p 2) _>>>
         [p (+ p 2)]]))

So what it does is matching of such elements (named p) that the next element (see cons) equals p+2, which would be a twin prime, since p is a prime. Then it generates an element consisting of a pair [p, p+2]. The pattern apparently matches such a list comprised of elements (see <join ... >) that are [p, p+2] (the inner <cons ...>).

You definitely remember that Lisp lists are singly-linked lists, and (cons head tail) prepends a new head element to the list tail. This is why it is matched, pertty similarly to how it's done in Haskell.

  print [(x,x+2) for x in primes if (x+2 in primes)]

  print [(x,y) for x, y in zip(primes, primes[1:]) if x+2 == y]

     my @primes = (1..∞).grep: *.is-prime;
     my @twin_primes = map { ($_, $_+2) }, @primes.grep: (*+2).is-prime;

infinite symbol gives away 6

So, yeah. Correct me if I'm wrong.

A slightly less listy pattern style is to declare types. Like I'd prefer this sort of declarative style:

     subset Prime of Int where *.is-prime;
     subset TwinPrime of Prime where (* + (2|-2)).is-prime;
     my @twin_primes = grep TwinPrime, ^Inf;

How is performance on a task like this compared to, say, Python?

Pattern-matching would be even more obvious is a piece of Haskell:

    pair :: [x] -> Maybe (x, x)
    pair p:q:rest = if q = p + 2 then Just (p, q) else Nothing
    pair _ = Nothing

    findTwins primes = [x | x <- map pair primes]

    import Data.List
    primes = Data.List.nubBy (\a b -> gcd a b > 1) [2..]
    findTwins primes = [(p,q) | (p:q:_) <- tails primes, q == p+2]
    main = mapM_ print . take 10 . findTwins $ primes

    findBigTwins gap primes = [(p,q) | (p:qs) <- tails primes, let q = p+gap, q `elem` takeWhile (<= q) qs]

The power of Egison's pattern matching becomes more obvious when we consider more general pattern such as (p, p+ 100) not only (p, p+2).

We can write the first 5 prime pairs whose form is (p, p+100) with a small modification for pattern matching for twin primes.

  (take 5
       (match-all primes (list integer)
         [<join _ <cons $p <join _ <cons ,(+ p 100) _>>>>
          [p (+ p 100)]]))
  ;=>{[3 103] [7 107] [13 113] [31 131] [37 137]}

Not only that, but you have redundancy between the ,(+ p n) part of the pattern and the emitted result of [p (+ p n)].

Point being: I applaud the attempt here, but it's not something I'll ever want to use.

Not if primes is a set.

Why should I write (+ p 2) when most people think (p+2)? This difference gets worse with more complex expressions. For the masses (and I mean a lot of very capable people) this is just garbage. For some it's wonderful. I can't say it's wrong, just not my thing. Perhaps it's the Blub paradox - I've never had the LISP epiphany so I can't say. I have had the python epiphany, and it's quite the opposite of this.

In Lisp you actually don't write parse trees. Lisp uses a serialized hierarchical data format of a tokenizer. S-expressions are not Lisp syntax trees, but a data format.

Someday we'll resurrect it all...

Reader macros blew my mind when I first read about them. Being able to change the very laws of the universe from within my simple little programs... Insane!

Actually, no - it's the other way around: more complex expressions really benefit from the uniform of syntax and using macros to simplify the code.

Not all macros are worthwhile, and some are detrimental to readability (and not writing those is one of the first Lisp lessons), but there are also macros (syntax extensions), which are elegant and powerful.

The common example is the `->` or `thread-first` macro, which chains a list of function calls, in the following manner:

    (-> (list 3) (prepend 2) (prepend 1) print)  ;; -> '(1 2 3)
    

    (print (prepend (prepend (list 3) 2) 1))

But it doesn't end there - this is just one macro, out of many powerful syntax extensions, which make for much clearer code. There are macros for lazy evaluation, both sequences (like generators) or expressions (more like `lazy` in OCaml); for partial evaluation of functions, for changing name resolution policy, for declaring classes, for looping, for pattern-matching, for error handling, for FFI, for logic statements, for transactional memory, for async control flow, and so on and on.

Some of these are trivial to implement, but some are hard or just complicated, with many cases covered; so you're not necessarily expected to write them yourself. If a macro's semantics are well-defined - as in most popular examples - importing a package and reading docs is enough to use them to a good effect. You don't really have to use any of them, but they all exist to make certain patterns in code simpler, more readable, and more convenient to work with. Other languages often lack many of them.

Anyway, what I want to say is that the more complex the problem, or the larger codebase, the more readable Lisp becomes, at some point surpassing most other languages. This is what makes Lisp fans to be so good at "recursion and condescension", and is also something you can't see in a `(+ p 2)` kind of examples.

just an observation. most people don’t think this naturally. most people learned (p+2). maybe there’s an argument for whatever is more natural, but the point is that people forget that what they view as natural was once unnatural and had to be learned.

as a small anecdote, i did a calculator competition in high school. we used those hp rpn calculators. it was unnatural at first but then i flew once i gained the skill in using rpn.

Lisp has plenty of syntax, it's just extensible and largely defined in operators. Commonly used macros like DEFUN, WITH-OPEN-FILE, and LOOP all add their own syntax to the language; that's what macros are for. When you put them together, typical Lisp code ends up looking like

    (defun count-words (filespec)
      (with-open-file (f filespec)
        (loop with counts = (make-hash-table :test 'equal)
              for line = (read-line f nil)
              while line do
                (loop for word in (cl-ppcre:split "\\s+" line) do
                  (incf (gethash word counts 0)))
              finally (return counts))))

    def count_words(filespec):
        with open(filespec) as f:
            counts = collections.defaultdict(int)
            for line in f:
                for word in line.split():
                    counts[word] += 1
            return counts

> Why should I write (+ p 2) when most people think (p+2)? This difference gets worse with more complex expressions.

As mentioned in other comments, there are infix packages available for when you're writing math-heavy code (or if you just need infix operators), but I also think that a lot of people don't write very much code with complicated inline arithmetic in it, and overstate how much the awkward math syntax would affect them (there are of course plenty of projects that are math-heavy and where it does make a large difference). When you're defining variables/functions/classes/etc or calling functions or looping or whatever, there's not much difference between Lisp and most other languages aside from whether the bracket is { or ( and which side of the keyword it goes on; in my experience, that sort of code tends to make up the large bulk of most codebases in most languages.

edit: Another place where Lisp's default syntax is significantly different from a lot of languages is something like object.f(x).g(y).h(z), which in Lisp looks like (h (g (f object x) y) z). This can be remedied with common macros like -> that let you write (-> object (f x) (g y) (h z)).

    IDictionary<string, int> CountWords(string f) => File.ReadAllText(f).Split().ToLookup(w => w).ToDictionary(kv => kv.Key, kv => kv.Count());

    let wordsCount f = File.ReadAllText(f).Split() |> Array.groupBy id |> Array.map (fun x |> fst x, (snd x).Length) |> dict

edit: And your C# version is only 24 non-whitespace characters shorter than the Python version, and 20 of those are because you called your variables 'f' instead of 'filespec' and 'w' instead of 'word'. A 4 non-whitespace character difference makes it exceedingly long and verbose? Or are you just advocating 1-letter variable names and avoiding newlines?

  (defun count-words (filespec)
    [(opip file-get-string
           (tok #/[^\s]+/)
           (group-reduce (hash) identity
                         (do inc @1) @1 0))
     filespec])

I posted this yesterday but deleted it, because grandparent's point wasn't about code golfing but just comparing Lisp and Python syntax.

There is a group-by function in TXR Lisp, but group-reduce is more efficient, because by using it we avoid building the group lists and counting. It's something I invented. Basically it performs multiple left folds in parallel, using the entries in a hash as multiple accumulators. Items from the sequence are hashed to their respective accumulator entry and injected through it. 0 is the initial value for the accumulator when it doesn't exist, functioning exactly like the initial value in a regular fold. (do inc @1) expands to a function which just increments its left argument (the accumulator) and returns it. We cannot use succ because it takes exactly one argument.

group-reduce has an added flexibility in that it doesn't construct a new hash, but takes an existing one as an argument. This adds the (hash) verbosity to the code, since we have to construct the hash ourselves. But with that we could run multiple successive group-reduce jobs that go into the same hash table. Also, the function is spared from having to provide a way to pass through hash arguments for different kinds of hash tables with different options.

The identity argument is needed because group-reduce takes a function that projects the items to keys; in this case the items themselves are the keys so we use identity.

Here is an interactive gist of how to solve the problem succinctly using group-by and then counting lengths:

  This is the TXR Lisp interactive listener of TXR 198.
  Quit with :quit or Ctrl-D on empty line. Ctrl-X ? for cheatsheet.
  1> [group-by identity '(1 2 2 2 3 3 3 3 3 4 4)]
  #H(() (1 (1)) (2 (2 2 2)) (3 (3 3 3 3 3)) (4 (4 4)))
  2> [hash-update *1 len]
  #H(() (1 1) (2 3) (3 5) (4 2))

  (defun count-words (filespec)
    [(opip file-get-string
           (tok #/[^\s]+/)
           (group-by identity)
           (hash-update @1 len))
     filespec])

> verbose and painful to understand.

Doesn't seem honest. I don't know Python, yet I can understand what that is doing, and might even be able to spot a logic error, if it had one (though not some issue of syntax).

People freak out about the parens. But I think what's the big deal? Functions are called with the paren in front of the function and the args space separated. so f(x, y) in C like syntax becomes: (f x y) - Whoop dee do, not so hard.

And since the syntax is extremely consistent, things that are math "operators" in other langs are just functions in lisp.

For "(+ 1 2 3)" + is actually the name of the function, so we are invoking + function and 1 2 3 are the arguments. Returns 6.

Once you know that, and that function args are evaluated before passed into the function, then you no longer have to worry about operator precedence and parenthesis to control order of operations. Once you get used to it, it's actually a lot simpler.

(* 2 (+ 5 5)) => 20

Actually not. Lisp has several types of forms. A function call is one. There are atleast two others: macro forms and special forms.

function form with +

  (+ 1 2)

  defun function-name lambda-list [[declaration* | documentation]] form*

The third type of form would be built-in syntax. LET is an example. The syntax for LET is:

  let ({var | (var [init-form])}*) declaration* form* => result*

  (let ((a 1)
        b)
    (setf b a))

This is not a valid Lisp program, which Lisp will complain about:

    * (let ((a 1 2)
           b)
      (setf b a))
  ; in: LET ((A 1 2) B)
  ;     (A 1 2)
  ;
  ; caught ERROR:
  ;   The LET binding spec (A 1 2) is malformed.

Is Lisp syntax easy? Once you look deeper, it actually isn't. Only a core language with only function calls is relatively easy.

* Lisp writes programs on top of a data structure called s-expressions. S-expressions are easy, Lisp not.

* Lisp usually has the operator as the first element of a lisp -> uniform appearance.

* Lisp has some built-in syntax. Usually as little as possible.

* Lisp lets the developer add new syntax via macros -> zillions of macros and the user needs to learn a few patterns to understand Lisp code.

If you squint a bit, macros and built-ins are similar in structure to functions (they're all s expressions).

I agree with much of what you said, but I still maintain that lisp syntax is still relatively trivial compared to most languages.

I also don't think exposing newcomers to macros on a language that is homoiconic is terribly useful until they're already comfortable with the basic s-expression syntax.

Because they have a parenthesis in front and back and the operator as first element. Macros then implement complex code structures between those:

  (foo a b)

  (let ((a 10)
        b)
    (declare (type (integer 0 100) b)
             (type number a))
    (declare (optimize (speed 3))
    (declare (special a b)
             (dynamic-extent a))
    (prog ()
      start
       (setf b (+ a 10))
       (go end)
      end)
    (the integer (+ a a b)))

Macro forms have lots of internal structure. For an extreme example check the syntax definition of the LOOP macro. Two pages of EBNF syntax declaration. That there are an opening parentheses and a closing one does not suddenly remove the syntax:

   loop for i from 1 below 10 by 2
          and
        for j from 2 upto 50 by 3
        when foo(i, j)
          collect i into is and j into js and i * j into ps
        when bar(i) > baz (j)
          return list(is, js, ps)

  (loop for i from 1 below 10 by 2
          and
        for j from 2 upto 50 by 3
        when (foo i j)
          collect i into is and j into js and (* i j) into ps
        when (> (bar i) (> baz j))
          return (list is js ps))

> lisp syntax is still relatively trivial compared to most languages

Not really. Check out the concept of a code walker in Lisp. That's a tool which understands Lisp syntax and can walk over Lisp code and do transformations.

Here is one:

https://gitlab.common-lisp.net/cl-walker/cl-walker/tree/mast...

Mildly complex... complexity comes in, because Lisp has built-in syntax transformations via macros.

Though stylistically my personal preference is for the macro to remain as simple as possible with limited syntax introduced.

With a decent macro system you can implement a type-checker (CL, Clojure and Racket do), as sophisticated as you want. But you can also write a type-level interpreter (of Lisp, if you want) which would evaluate programs during compilation (I can't find the post anymore, someone was describing their job interview gone... weird...). The difference between the two, more than with other such comparisons, comes down to aesthetics and cultures.

IOW, you can't infer the lack of ordering based on two items having an ex aequo position.

I guess you could say that jargon is just another form of syntax.

But I didn't say that, because I thought it was a bit of a digression to my point, which was already a digression on phkahler's point, and the digressing has to end somewhere...

I’ve also noticed that it “pivoted” to focus more on math; is that intentional?

Anyway, good to see it on HN again. Anything that pushes more pattern matching research is a plus for me =). Mainstream languages barely started to adopt first-order pattern matching!

I started to implement a computer algebra system as a killer application of customizable pattern matching of Egison.

It allows users to customize the pattern-method for mathematical expressions from more primitive level than the other computer algebra systems.

I doubt I have time to learn this or use it but it would be cool to see a list of examples of things that are much more concise and elegant in this language so that it expands my thinking. Is there a list of examples with comparisons to traditional languages?

Here is a link for a new paper that discusses the integration of tensor index notation including the support for differential forms.

https://arxiv.org/abs/1804.03140