> a big difference that Common Lisp brings to the table is that it has separate namespaces for variables and functions. This may not sound like much but it makes it much easier to write macros (code that writes code) and facilitates compile-time computing.

A little more detail here: it has a lot more namespaces than that. Indeed, you can attach arbitrary different namespaces of things to symbols if you want to.

It turns out that when you're writing macros in Lisp, the majority of what you use from outside the macro are functions and the majority of symbols that you bind and use within the macro are variables, so if you put them in different namespaces, you get rid of the majority of cases where symbols don't do what you were expecting in a macro.

Now, it's not quite all of the cases, and when it does happen, it's a pain to track down. Which is why the Scheme folks spent the effort and complexity on hygienic macros (where the system rewrites all symbols internally to make variable capture impossible). At the time of Common Lisp's creation, multiple namespaces and unhygienic macros were the obvious choice for production code, since there weren't hygienic systems available. Today you can go either way. In terms of research, hygienic is the route to go, as the work over the last couple of years on strongly typed, hygienic systems has shown.

> A little more detail here: it has a lot more namespaces than that.

And I don't think the function/value split is even the most important namespacing it does with regards to macro hygiene ("functions are less likely to be shadowed" always struck me as a kind of weak argument). Symbols being namespaced under packages is a much more robust solution:

    CL-USER> (defmacro m (x) `(car ,x))
    M
    CL-USER> (defpackage :p)
    #<PACKAGE "P">
    CL-USER> (in-package :p)
    #<COMMON-LISP:PACKAGE "P">
    P> (cl:macroexpand '(cl-user::m x))
    (COMMON-LISP:CAR X)

CL:CAR isn't accessible in P, so (car x) by itself would signal UNDEFINED-FUNCTION, but the macro works correctly. Defining a P::CAR function wouldn't change anything.

That isn't just for functions, either, eg if you do:

    (defmacro awhen (pred &body body)
      `(let ((it ,pred))
         (when it ,@body)))

(where you want to leak IT), and then import AWHEN but not IT, you'll get an error that YOUR-PACKAGE::IT isn't bound to anything when you try to use it; it doesn't matter that AWHEN-PACKAGE:IT is bound.

I don't disagree with anything you said, I just think it made CL macros look flimsier than they are.

The problem is that it is quite common for cl:car to be accessible in packages, because of the practice of use-ing the cl package everywhere.

Even if I'm in my own package, I can't do this:

  (flet ((car (x) ...)))
    (m arg))

because in order to have a hassle-free Lisp coding experience in my package, I brought in all the public symbols from the common-lisp package. So my flet is in fact shadowing cl:car.

Even if I just import the specific common-lisp things I need, and car is not one of them, that could change. Today, that car above is mypkg:car. Tomorrow, someone edits the defpackage to import car (because they needed it in a function they added), and now when that file is re-read, car is cl:car.

Packages have a theoretical solution to the hygiene problem that will not be air-tight in practice due to use/importation.

Another problem is that programmers aren't going to define a large number of packages to protects parts of their program from each other. A common practice is just to make one package for an entire project.

You need fine-grained package use to achieve near perfect hygiene. Ideally, each module that provides macros should have its own package, and use only symbols from that package in the macro expansion. If module A uses a macro from package B, and that macro generates a local function F, that will be B::F, not interfering with the A::F function. If the modules are in the same master project package, the clash is not averted, obviously.

The function and variable namespace difference is a matter of taste to me. I prefer naming my variables lst to having to use #' and funcall, and the extra fuzz that hygienic macros bring is generally not a very big problem once macros become complex enough.

What CL brings over scheme is a fantastic object system. The Scheme copies are half-assed in comparison, especially considering how well-integrated CLOS is.

CL success stories are growing a bit long in the tooth. Sort through the links yourself to get an idea. http://www.lispworks.com/success-stories/index.html

The best CL implementations unfortunately are not opensource. There is still a large use of Lispworks by many heavy CL developers. And anyone who has used it knows what they have to offer over the opensource equivalents. Support is something well worth its cost. Plus in production things tend to just work on Allegro and Lispworks. They tend to bench well for the things we need. http://ftp.linux.org.uk/~ober/report.html

Hadn't heard of ITA (authors of QPX before being bought by Google) before. Now that I'm looking, it isn't clear to me QPX is still written in Common Lisp? Has anyone confirmed recently that it is still a big CL project?

I hired/partnered with one of their former engineers. QPX is most certainly still written in Common Lisp.

Thanks for the confirmation!

They also published some code: https://common-lisp.net/project/qitab/

>much easier to write macros (code that writes code) and facilitates compile-time computing

Isn't this a non-issue in scheme as it has hygienic macros?

Note: my only exposition to lisp is via elisp, so I might not know what I'm talking about.

Most CLers hate scheme macros since they almost uniformly believe that syntax-rules is the only macro facility. I myself have implemented defmacro using syntax-case because it is sometimes nice to have.

Hygiene solves the issue of namespacing macros, but at the cost of a very complex implementation. Implementing hygiene is HARD and I don't think it can even be called a solved problem. The current systems have issues that have proven to be hard to resolve.

That's intersting, because the lisp 'the right thing' approach [1] calls for correctness and ease of use over simplicity of implementation.

[1] as opposed to the alledged 'worse is better' of the unix world.

It is a matter of preference. Sometimes you end up using gensym in scheme as well, especially for introducing identifiers in bulk.

> uniformly believe that syntax-rules is the only macro facility

far from it. CLers know that Scheme has zillions of different macro systems. Which makes the whole topic highly confusing and intellectually interesting.

What I meant by that is that CLers usually complain about hygiene as if it was impossible to break. I don't know how many times I have had to explain that syntax-rules is the high level macro facility and that all schemes provide their own lower level macro facility, be it explicit/implicit renaming (chicken), syntax-case (r6rs) or syntactic closures (Chibi?).

I understand the complaint that this isn't portable. It isn't. R6RS tried to standardise it. Hopefully r7rs does better in that regard.

> that all schemes provide their own lower level macro facility and that all schemes provide their own lower level macro facility ...

or the equivalent of Lisp's DEFMACRO. There are tons of options. But it's a bit disappointing that such a central feature hasn't found acceptance of a main solution in the Scheme world. R7RS small has only syntax-rules AFAIK, and something like syntax-case is planned (?) for the R7RS large (which does not yet exist, AFAIK). It's not even clear to me whether 'syntax-case' is universally seen as 'solved problem'.

Anyone who has a basic idea of Scheme should know that syntax-rules is not the only option - a short look into a manual of an existing Scheme implementation should be sufficient...

Not quite, the low level macro facility provides hygiene as well (well, maybe not explicit renaming macros) with the ability to break hygiene when desired. These systems can be used to implement an unhygienic macro system (iirc, my defmacro about 7 lines, including the syntax case boilerplate).

R6RS standardised syntax-case, so any r6rs scheme already has it, but due to many things r6rs was a huge controversy and many implementations never started using it. Guile, chez, racket, Kawa, jscheme, larceny all support it. I expect syntax case in one form or another to make it into r7rs large, but you never know. Lots of cool things are making its way into r7rs large, including HAMT-based maps. There must of course be a vote, but that is just too good to not have :)