1. Documentation

Racket has a beautiful documentation system called Scribble [0]. Almost all documentation written for Common Lisp is ugly, inconsistent, and doesn't follow any sort of standard. Common Lisp has a gazillion "documentation generators" which do little more than slurp up all the documentation strings and barf them back out as e.g. HTML. Common Lisp users tend to just rely on READMEs + jump-to-definition to learn how a project works. A few Lisp programmers (e.g., Shinmera, Edi Weitz) have made their own documentation systems that look better than average, but are still at least an order of magnitude less capable than Racket's offering.

Racket's documentation system is aesthetically pleasing, very well organized, and makes the programmer want to write long-form documentation. Moreover, Racket (the language) sets a superlative example of what its users can strive for in terms of quality of clear, no-nonsense writing.

2. Languages and DSLs

One of Lisp's (any dialect) defining features is the ability to build domain-specific languages, syntactic extensions, and the like by way of a special kind of function that can run prior to the code's execution. These special functions are called "macros", and across Lisp dialects, there are lots of different takes on them.

Common Lisp gives you a grab bag of facilities for writing code that lets you change syntax at different levels: DEFMACRO (scoped variant: MACROLET), DEFINE-COMPILER-MACRO, DEFINE-SYMBOL-MACRO (scoped variant: SYMBOL-MACROLET), SET-MACRO-CHARACTER, and a few others. At best, I'd consider these only loosely related, and they all work on really "raw" data representations (parsed S-expressions and character streams). In part because of this, ANSI Common Lisp doesn't provide access to any notion of environment, source location, etc. when writing new macros.

Macros defined by the above operators "live" in different places and are compartmentalized in different ways. There are some Lisp libraries that help you tame definitions and keep them fenced in, like NAMED-READTABLES which helps you organize where reader macros (macros which change how Lisp is parsed at the character-level) live and how to turn them off and on in a sane way.

Contrast with Racket which has explicit documentation on how to write macros [1,2], how to debug them [3], and how to encapsulate them a new DSL embedded in the system [4]. Racket—as a technical system—clearly defined the phases of compilation, what objects are manipulated during the phases (e.g., "syntax objects"), how to provide debugging information, and so on.

In the end, Common Lisp's system for making DSLs feels a little runny. It's supremely hacker-friendly, but it's a bit more difficult to deliver, in one fell swoop, a language as a library. Racket, on the other hand, at the expense of having increased rigidity and needing to read two or three complete chapters of a manual, you get a way to neatly bundle up a language. This is why Racket had nice DSLs like Typed Racket, Hackett, Scheme RnRS, etc.

I use Common Lisp and I develop relatively large DSLs in the language, and even if I want to, it's hard—if not impossible—to write large DSLs that have nice error messages, source locations, etc. without writing an entirely new "reader" (lisp's term for a code parser) myself every time. Smaller DSLs where user code doesn't exceed more than a few lines? Examples like:

- regex syntax

- string interpolation syntax

- a pattern matching feature

Common Lisp is great, and you get a ton of mileage out of the facilities it provides. But larger ones? Examples like:

- an embedded language for writing documentation with markup

- a complete language with lazy evaluation semantics

- a language for non-deterministic programming (like SCREAMER, Prolog)

- a language for strictly typed functional programming (like Coalton, Hackett)

Tackling these in Common Lisp requires ambition and a very steady hand, whereas Racket's overall system (language, implementation, documentation) makes that a comparatively painless exercise.

[0] https://docs.racket-lang.org/scribble/

[1] https://docs.racket-lang.org/guide/macros.html

[2] https://www.greghendershott.com/fear-of-macros/

[3] https://docs.racket-lang.org/macro-debugger/index.html

[4] https://docs.racket-lang.org/guide/languages.html

This is not what you'll usually read in forums, I think DEFMACRO is the better design here.

Or, import defmacro [2] and use that.

As can be seen, Racket actually provides more power and choice in this regard, including support for Common Lisp's way.

In any case, I make no claim to either being better or worse. I just think Racket is more principled in its mechanisms for defining and delivering a DSL to a user.

[1] https://docs.racket-lang.org/guide/proc-macros.html

[2] https://docs.racket-lang.org/compatibility/defmacro.html

BTW, Racket's defmacro is only partially compatible with Common Lisp defmacro. It doesn't really work. I even seem to remember (? from long ago) that it's impossible to implement a genuine CL defmacro in Racket.

not totally wrong but i did write this https://github.com/disconcision/fructure with ~2 years programming experience. implementation features some nice simple define-syntax-rule macros which i maybe never wouldve managed with defmacro

The `defmacro` model is "simpler", true - but it is fragile and breaks down when you want to do complicated things.

Check this blog post written by someone, who knows hos Common Lisp.

EVAL-WHEN considered harmful to your mental health https://fare.livejournal.com/146698.html

Though I concede this case could be analogous to trying to write a Doubly Linked List in Rust, it's not enough to say definitively yea or nay. So I'd more like to see a concrete case you have in mind where the tradeoffs are squarely against CL. Like, the loop macro is more complicated than any I have written myself, but you can break it down, and it's not that bad -- I think Norvig's version is pretty neat to study: https://norvig.com/paip/loop.lisp This project (and it's not the only one!) adding C syntax to CL https://github.com/y2q-actionman/with-c-syntax I think is more complicated than loop, and is sort of where I'd put the level of "complicated things" at that I'd like to see an example from the Scheme world that clearly shows defmacro's deficiencies on some metrics. (Fewer bugs? Easier to add new features to? Shorter code? Faster performance either at compile time or runtime or both? Easier to understand or faster to implement for people with similar levels of skill in the language?)

    (define-syntax (nest stx)
      (syntax-case stx ()
        ((nest outer ... inner)
         (foldr (lambda (o i)
                  (with-syntax (((outer ...) o)
                                 (inner i))
                    #'(outer ... inner)))
                #'inner (syntax->list #'(outer ...))))))

Note that the `nest` form just rearranges its input - so there is no problems getting the scope of identifiers correct. That is: we are well within the comfort zone of `defmacro`.

With regards to the `syntax-rules` solution: Fare refers to "when I learned Scheme" and pre-R6RS there were no `syntax-case` in the specification. In practise all implemenations did have macro systems more expressive than `syntax-rules`, but it was fun to see whether something was possible with plain `syntax-rules`.

The "special sauce" is being explicit about compile time vs runtime (aka using phases).

The paper "Composable and Compilable Macros: You Want it When?" by Matthew Flatt describes why conflating runtime and compile time is a problem.

http://www.cs.utah.edu/plt/publications/macromod.pdf

    #lang racket
    
    #;(defmacro nest (&rest r)
        (reduce (lambda (o i) `(,@o ,i)) r :from-end t))
    
    (begin-for-syntax
      (require racket/match)
      (define (reduce f xs)
        (match xs
          [(list x0 x1) (f x0 x1)]
          [(cons a d)   (f a (reduce f d))])))
    
    (define-syntax (nest r)  
      (reduce (λ (o i) #`(#,@o #,i))
              (cdr (syntax->list r))))
    
    (nest
     (list 'foo)
     (list 'bar)
     (list 'baz)
     'qux)

And Racket has syntax-parse, which is even nicer, I seem to recall.

We have to go deeper! Could you write a DSL in CL to help you write new DSLs as easily as in Racket?

Nevermind then, turnstile is still something fitting for this topic : )

I would go as far as to say that a non-negligible part of why people are intimidated by Rust is the rustdoc format and layout. Thankfully it makes up for it with an incredible Rust Book.

To me the gold standard of docs is Elixir followed by Racket. Even Go's docs are better than Rust, but that's mostly because of a much simpler type system, so simpler autogenerated docs for structs and their methods.

Scribble beats rustdoc hands–down when it comes to authoring documentation, but even with Rustdoc you can do a good job of it if you try (http://db48x.net/reposurgeon/rust-port-docs/reposurgeon/path...). I’ll have to check out Elixir next.

Here's an example of Elixir docs: https://hexdocs.pm/ecto/Ecto.html

Note the layout, the amount of prose to introduce a module and the function documentation that is not limited to just a quick line but often has multiple examples and tries quite hard to be clear and extensive.

There is no reason Rust docs couldn't be that good, but the example has to come from the top: improve the std docs, and everything will follow suit.

As a new Rust fan I would just love to have the std docs half as good as Elixir's. I constantly get lost in rustdoc.

I'm afraid, much Go code doesn't have that much structure. At least none that you could express in a machine readable form.

    @tabular[#:sep @hspace[1]
             (list (list @bold{Animal} @bold{Food})
                   (list "mouse"       "cookie")
                   (list "moose"       "muffin"))]

And the table of contents promises literate programming as well. Someone was really thinking when they put this together; I really like it.

The documentation generated for individual functions (such as <https://docs.racket-lang.org/scribble/base.html#(def._((lib....>) is fairly nice as well. Readable, linked together by hyperlinks on the type predicates as well as hyperlinks in the body of the documentation itself; it’s pretty good. Rustdoc puts links to the source code for each one though, and I would soon miss that.

These sorts of languages/language extensions pop up all the time in "ordinary" programming. For example, common DSLs we see often.

- regex is a language for string patterns

- JSON is a language for simple nested data

- SQL is a language for expressing DB queries

- OpenMP contains (essentially) a language for parallelizing loops

- (this might sound crazy because it's so common and basically built in standard in almost all programming languages:) infix notation for math

The problem is that many of these are absolute kludges in existing languages. Anybody who has had the displeasure of trying to interpolate DB table names into SQL SELECT strings knows this. Most languages try to make it a bit better by adding preprocessors, using builder-patterns, abusing reflection, or otherwise. Looking under the hood of some languages' implementations of these ideas will often lead you to see the most cursed of programmer things in existence.

Each of those languages (or close syntactic cousins of them) can be written directly in Lisp, as a library, without shoving them into strings, or using external tools, or making custom file formats. They call can exist as a part of the Lisp language as a syntax extension.

Making DSLs requires exercising that muscle a bit; it's non-trivial to design a new and useful language that fits orthogonally into an existing system.

It's like being introduced to OO programming where, at least for me, it made sense but it wasn't immediately obvious what a good object model for a problem might be. It took me lots and trial and error to understand how OO works in "the real world", where it's good, and where it falls short.

Once you get they hang of making DSLs—or even just small but powerful syntax extensions—it's an amazing tool to have at your disposal.

Maybe a simpler example, what if I wanted this in (pseudo-)C?

    with(s = fopen("/foo/bar", ...)) {
        ...;
        ...;
        println(readln(s));
        ...;
        ...;
     }

In Lisp, this is a feature you could build and provide as a library feature. In fact, we can do it, straight here in an HN comment.

    (defgeneric destroy (x)
      (:documentation "A destructor for use with the WITH feature".))

    (defmacro with ((var = val) &body body)
      "Python-style RAII macro. VAR will be
       bound to the value of VAL, and upon
       exiting scope, will be destroyed with
       DESTROY."
      (assert (eq = ':=))
      (let ((orig (gensym)))
        `(let* ((,orig ,val)
                (,var ,orig))
           (unwind-protect
               (progn ,@body)
             (destroy ,orig))))

    (defmethod destroy ((s stream))
      (close stream))

Now, we effectively have RAII in Lisp:

    (with (s := (open "/foo/bar" ...))
      ...
      ...
      (write-line (read-line s))
      ...
      ...)

This is a concept we can use almost immediately after learning about it, instead of waiting for Python's PEP 6363 to pass with consensus from people who may have different programming goals than you do.

https://github.com/norvig/paip-lisp/search?l=Markdown&q=defm... - all references to defmacro in the markdown files

Chapter 3 shows a simple macro, just adding a while loop to the language.

Chapter 9 shows some more complex ones, including a with- macro and a grammar compiler macro.

Chapters 11 and 12 show the development of a Prolog implementation in CL using defmacro to aid in compilation again in Chapter 11.

Chapter 12 shows adding an OO system to the language. Technically not needed with CLOS, but a good demonstration of what can be done with macros.

There are other examples (why I included that search link). Macros let you change the language in ways large and small. Many uses could probably be replaced with functions, though you'd end up having to throw a bunch of quotes about or closures in order to delay processing things. You'd also be delaying that to runtime, which incurs its own penalties compared to macro expansion and compile time (if a compiled CL).

A well written function must also abide by some promise or specification—contracts on the inputs and invariants on the outputs. I don't see why new syntactic structures are much different from that.

I think there's undue spookiness around macros among lots of programmers, especially those who haven't sought to solve some problem with them.

- "It's too powerful for the common person."

- "It's way too likely to make code confusing."

- "It's invariably too hard to reason about."

- "It won't work on a team because everybody is going to make their own weird incompatible language and it's just not scalable."

All of these sentiments are complete bunk, and I argue are against common wisdom around other programming abstractions, like functions or classes.

And a lot of what tends to be macros in Lisp turns into functions in eg Haskell. Eg you can define your own new branching and looping constructs (like 'if' or 'while') etc as functions in Haskell, but in Lisp these need to be macros.

In Lisp, you can also define branching and looping as functions, but you need to concede and have the arguments themselves be (lambda) functions.

    (defun while (c body)
      (when (funcall c)
        (funcall body)
        (while c body)))

Yes, if you wrap your arguments in lambdas, you can write loops like that.

Though to be as performant as the built-in constructs, you need a rather clever compiler that 'removes' the lambdas again.

I wrote a post about this here[1]

[1]: http://tech.perpetua.io/2022/01/generating-sqlite-bindings-w...

Quicklisp has become the main way many (most?) people handle installing a library for Common Lisp. Under the hood, ASDF is used for defining the system and specifying how it should be compiled and such as has also been standard in the community for a very long time.

If you see a library you want to use, like the HTTP client drakma, then you just start your Lisp and write

    (ql:quickload "drakma")

Considering them must-haves is perfectly reasonable for a programmer who wants to work in that style. For example, being able to (put roughly) modify any part of the system at any time, and let the semantics decide what happens.

Scheme/Racket absolutely is not CL, and Racket has been moving further away from CL than Scheme already was. I've worked on critical production systems in both Scheme/Racket and CL professionally, and they are very different languages, and people wouldn't confuse them if they didn't both use S-exp syntax. (Individual Lisp family developers are also different. There are some shared memes, but you don't end a Lisp family language expert today, and shunning that $500K Python job, without being stubbornly self-motivated and/or self-destructive.)

One of the keys to learning Scheme originally was to put aside my biases about what was important, try to use and then understand the idiomatic way, so that you can then decide what makes sense.

(Examples: My own first bias when learning Scheme was trying to micro-optimize for runtime as I coded, like I would in C, but I didn't know what cost model I should reason about. If you look at the first Scheme code I wrote, https://pkgs.racket-lang.org/package/html-parsing , it's a little like a C programmer with some non-Scheme Lisp experience.) One of the next barriers was learning to avoid mutations and premature exits, and to leverage tail recursion. Before I learned that, I couldn't have told you that can make code easier to read.)

When we were first learning computer things, there was an implicit suspension of disbelief or acknowledgement of not knowing. As we grow more knowledgeable, we have to be careful not to plateau or become rigid, assuming we know the right way, and dismissing things that are nonintuitive. Sometimes we will be right to dismiss them (and often will be, in corners of "tech"), but sometimes we will miss out. It's even not necessarily a net loss if we invest in understanding a perspective, and then determine its flaws, because we may still have added some new useful ideas to our mental toolkit.

But, no worries: Haskell programmers still have plenty to rant about Python. And vice versa.

You either have to give them a name (with 'def'), or you only get a restricted subset of syntax (with 'lambda').

In practice, giving them a name is not too much of a hassle.

Racket’s portable GUI and app builder are very good, as is LispWorks’.

Ehh.. they're pretty good and I'm thankful for them. I don't think I'd go as far as 'very good' though; the selection of widgets provided is basic. My rule of thumb is that if your application only needs the sort of functionality you can find somewhere in DrRacket, then racket/gui will be pretty good. But if you need a widget you can't find in DrRacket, then you'll probably have to implement that widget yourself.

Google employs Common Lisp programmers as well for their flight search product.

RavenPack, SISCOG, GrammaTech, Grammarly, SRI (?), MIND.AI, and others as well (or at least used to). Check out [1].

[1] https://github.com/azzamsa/awesome-lisp-companies

* — there are also Slyblime for Sublime Text and ALIVE for VS Code, but they are very beta-quality.

Kaveh's Common Lisp Lesson 01 https://www.youtube.com/watch?v=nSJcuOLmkl8

He doesn't take you by the hand to setup an environment, but it's a neat exposure (introduction isn't quite the right word) to someone using CL to do interesting things.

Mind its a bit Mac and Clozure CL specific if you wanted to follow along, but it's still a good watch of just seeing someone doing stuff without explicitly telling you what they're doing. Sometimes you just want to get stuff through osmosis, and he does interesting, basic things with CL at all sorts of levels.

The basics of the commands do things like send code over for compilation and redefining things, running things, responding to errors, debugging things, jumping to definition locations.. things you'd normally expect an IDE to do, except because of CL itself so much of that is part of the language and not a separate IDE. Some more editor-specific things come into play that some people really love, and while they may be best done in emacs they're not unique to emacs either. (Example: suppose you messed up the order of your arguments to a function, so it's written like (my-func (list something) some-other-arg) -- editors have support to swap those two arguments in one go, rather than having to backspace and retype or whatever. I personally don't find such features that magical, and are table-stakes even for things like Eclipse editing Java. Meanwhile even emacs is still weak for some things like large-scale automated semantics-aware refactoring...)

I'm fond of this short 5-part blog series https://malisper.me/debugging-lisp-part-1-recompilation/ (has some emacs gifs) that shows some of these features. But notably again, the experience isn't limited to emacs.

In other words, I never coded something sufficiently big to understand the pros of CL.

Hence, I'm curious. What stuff I'm missing out in terms of language features and libraries?

use common lisp (sbcl) if you want c-level performance

use common lisp if you want to make optimizing compilers

use common lisp if you want smalltalk type repl driven interactive development

The new Racket compiler was derived from the Chez Scheme compiler.

https://en.m.wikipedia.org/wiki/Common_Lisp_Object_System

On r/lisp there was a rant that included a reflection that things would be done in a third the time. There was an ask for $60M to apply Lisp to know a different world. Apple can easily afford a $180M experiment to see Ralph/Dylan implementation of iOS/macOS. If proved successful would the software platform leap how the hardware has done across multiple architectures? The ARM M series devices are a there and back again move given Apple Newton was on ARM and Apple invested which kept the idea alive.

https://download.clojure.org/papers/clojure-hopl-iv-final.pd...

The design decisions in this paper strongly resonate with me. It helped me to overcome my aversion to JVM-hosted languages, and settle down to learn Clojure.

I'm a research mathematician, at one extreme where code is steps up a mountain. I don't need to summit twice, so productivity is far more important than speed. Nevertheless, the fastest Clojure time trials use GraalVM:

https://www.graalvm.org/

There have been very substantial GUI applications implemented with both systems.

In the lisp tradition, they made a lisp geared towards their use case, specifically. It's called Scheme... which sports much more than 'a pseudo-macro system'.

A language does not necessarily become limited, when it is simple. If the right simple concepts are available, all kinds of stuff can be build with them, bootstrapping more complex concepts or features. It becomes rather a question of how much is already there, done by others, or how much work oneself want to put in to have some concept in ones language. So I wouldn't necessarily say, that it becomes a limited language.

Another part is stripping of useful features to ease implementation. The Scheme language is full of uncomfortable, low-level constructs for the sake of simplicity. In contrast, Common Lisp is far from simple but it contains many high-level constructs which are nonetheless simpler to use.

On the contrary. A simple language can be relatively quickly implemented on another system, because there is not much to implement. If we look at how many Schemes there are, implemented for various platforms, and compare that to how many Common Lisps there are, we quickly see that difference. Of course CL is also more geared towards being "the one" language, rather than "make your own version of CL", exactly because of the effort required to do so. In that way one could argue it cannot even get close to the portability of a simpler language.

> Another part is stripping of useful features to ease implementation.

With useful features, you mean things, which reduce work of implementing things? Of course a bigger language will have more stuff built-in – That's in the definition of bigger language. The point is though, that Scheme does enable you to build these things. Once there is a library for it, you can use it just like other in-built things in CL. Whether that always happens to happen (a library existing), is another question. There is nothing in the language, that would prevent you from building that thing though.

> The Scheme language is full of uncomfortable, low-level constructs for the sake of simplicity.

I find them very comfortable. I also find its simplicity refreshing and a pleasure to use. It enables me to gain more understanding of concepts, which I might implement or copy from others.

> Being easy to implement from scratch is a must for both uses cases.

If you are teaching how to write a compiler or an interpreter, yes. Otherwise I would have to disagree.

If we're talking middle school then there are some arguments to be made that Racket's easy-to-use graphics libraries are more conductive to learning. It will never be Flash though.

Not having a corporation back the development of a language has tons of downstream effects, including:

- lack of support opportunities

- lack of employment opportunities

- lack of evolution and development of an ecosystem

- higher churn and/or attrition

- fewer opportunity to do long-term design and implementation projects

Yes, sometimes there are FOSS miracle workers who are so extraordinarily productive without being paid, but they're few and far between.

(Aside from that, not all users need such functionality.)

Parallelism is one of those. Only microcontrollers are single-core these days, and not even all of them.

I sympathize with the people who have fallen in love with their own idea and are sad that others "don't see the light" and don't support them -- but our world is an imperfect system and that's a fact of life.

F.ex. I'd kill for a LISP with strong(er) static typing plus Erlang's transparent parallelism and runtime but yeah, we don't have that. So Elixir and Rust it is for me.

interestingly, the person you are replying to is an author of this little lib

https://coalton-lang.github.io/

Racket /also/ has `thread` which is a co-routine but afaik you cannot pass the co-routine off to another native thread and there are certain constraints passing data between different threads, similar to transferring ownership.

I am not aware of the SBCL ecosystem to comment.

HAXL

Lispworks/Allegro is better if specialized requirements or actual money is on the table. Lispworks supports mobile platforms IIRC which SBCL does not, for ex. But if all one needs is server side deployments or personal projects, there is no need, nor any use of commercial implementations, as good as they might be.

For a while I used SLIME as my debugger and vim as my editor.

As mentioned by others, its not. But Emacs is also a Lisp runtime. People who are attracted to Emacs eventually learn some Lisp (Emacs Lisp, closer to CL than Racket et al) or were attracted because of Lisp in the first place. As a result, the overlap between CL devs and Emacs users is naturally higher than that for other editors. This translates into people developing more and better tools for their editor, and higher popularity usually translated into more and better tools (with obvious exceptions).

As mentioned by other commenters, SLIME can be used with Vim too, if that's your thing. But just like SBCL, Emacs is kinda of de-facto editor of choice, not by force, but by love of its users :)

I used CLISP because it had GNU readline. (e) to edit my file, (l) to reload it, and off to the races (I wrote those trivial defuns). No packages, no imports, no asdf. Everything in CL-USER.

I did switch to emacs for one simple reason: tab to reindent. Other than that it was the most basic of emacs use (load file, save file, ^s to search, cmd-XCV for cut and paste, pgup, pgdn). No Sexpr navigation, no buffer juggling, none of that, no lisp integration whatsoever outside of paren matching and the tab to indent.

The language isn't, but SLIME is because emacs isn't just an editor, it's a platform and system that people develop applications on top of. And the community was already there, using emacs, to continue with that option.

It just happens that SLIME is a pretty well designed UI (the acronym stands for Superior Lisp Interaction Mode for Emacs, so I guess that was the goal). It’s miles ahead of what you get from most other languages, even C++, but most of what makes it so great comes from features that were already present in the language implementations. It also combines very well with other advanced Emacs packages such as Paredit.

Interestingly, SLIME is implemented as a network protocol to allow for remote debugging, so you can implement a new UI for it any way you want. I’ve never used it, but you might check out SLIMV instead. https://www.vim.org/scripts/script.php?script_id=2531

https://github.com/kovisoft/slimv

(Never used it myself so can't comment on it beyond noting its existence.)

The short version is I have SLIME running inside Emacs, and my text files open in vim. I would just reload the changed file(s) when needed (originally with cl:load-file, then later with asdf, which is roughly "make" for lisp). I used slime similarly to how I would use gdb. It was a big improvement over using vim with the clisp[1] REPL both because SLIME was more powerful than the clisp repl, and because it worked with other implementations like CMU CL (which sbcl now is a fork of).

I mostly just used the menus for interacting with emacs, but since the shortcuts are listed beside commands, I did learn the more commonly used ones.

I continued like this for years until I discoveredevil-mode[2], which was the first thing close enough to vim to be usable for me (and even then I had to add a few keybindings that it was missing).

1: clisp is a specific implementation of common lisp, not an abbreviation

2: https://github.com/emacs-evil/evil

It doesn't do everything, it doesn't do it as elegantly, but it works

https://franz.com/about/company.history.lhtml

i think it would be nice for someone to do a comparison of development in emacs+slime vs allegro cl vs lisp works

I haven't used the recent versions of ACL and LW, so I cannot make a direct comparison, but for the experienced user, Emacs offers a lot of productivity.

I would say it's just karma.

A lot of the cheaper commercial implementations did not survive or did not thrive: Corman Lisp (now open source, but not used much), Golden Common Lisp, Exper Common Lisp, Procycon Common Lisp (got bought by Franz), MuLisp, Macintosh Common Lisp, ...

Clozure CL is the result of a project extracting/porting the bare CL implementation from MCL. It was open sourced. It got a GUI later, but never again had the polish and appeal of a product like Macintosh Common Lisp.

I've thought about learning a Lisp in the past, but the Lisp community's insistence on using Emacs has completely put me off of it. I will not use Emacs under any circumstances, and you couldn't pay me enough money to change. And this had lead me to conclude that Lisp is simply not for me.

Anyway, not sure why anyone but you should care about you learning something?

Everyone have to deal with their own psychological barriers somehow.

Also I don't care about Vim bindings. I use Kate for writing code about 75% of the time anyway and would prefer to continue doing so.

I suspect a plugin for Kate could be made, but I didn't see one after a very brief search. The maintainers probably want to encourage more use of LSP anyway. There is an LSP server for Common Lisp, but it's pretty basic, and is still at odds with how you want to do things. (Perhaps Clojure has a more developed one.) It at least "works" with Kate in the sense that Kate successfully launches and talks to it after making a custom settings.json, and gives me simple symbol completion and docstrings on hover. But to learn about my own functions and their docstrings, I need to be able to load my code into the Lisp instance that's also running the LSP server. (Traditionally IDE-ish features in Lisp aren't done by external tools statically scanning files, but by introspectively asking the Lisp program itself for the info, which may include file info.) I did hack something together so that I was able to copy-paste some code from Kate to the split terminal panel running a second instance of Lisp that talked to the one Kate launched, but this isn't very productive. You could maybe get by with writing some code, saving, and then reloading the whole set of files (system) with one command when you want, but you'd need even more work to make errors interactively recoverable, which is another really nice feature of Lisp development when you have proper editor support.