> I can give you concrete examples of demonstrable >10x productivity improvements which resulted in products succeeding where they otherwise would undoubtedly have failed.
Well, I would definitely be interested in these examples. I occasionally write Lisp (admittedly, Emacs Lisp rather than Common Lisp) and while I appreciate having macros and other metaprogramming tools at my fingertips, I've never encountered a situation in which their use was critically important. I can always replicated the thing I wanted to do in Python with a bit of boilerplate; if I had to choose, I would certainly take Python's huge ecosystem over Lisp's metaprogramming. Frankly, I don't think there have been any language silver bullets after structured programming and garbage collection. So I'm very skeptical of the claims of extreme Lisp productivity. I'm open to being convinced otherwise though.
I think a couple factors are at play here. First, most developers never really learn metaprogramming or use it, even in languages with native facilities for it. You don't need it to get the job done, strictly speaking, and it is a difficult skill to acquire. Second, many software applications don't benefit that much from metaprogramming even when you have those skills. The benefits aren't universal, which brings the costs into question.
Nonetheless, for some types of software, writing code without using metaprogramming will have several-fold the LoC, complexity, etc of the equivalent with metaprogramming. But if you never developed metaprogramming skills, you are unlikely to recognize when these opportunities arise. In these cases, you do see large productivity multipliers. I see this pattern all the time in C++; most C++ developers have no idea how much concision (and type safety) metaprogramming enables in contexts where it is perfectly suited for the job because they never learned metaprogramming in C++, so they write vast amounts of brittle boilerplate instead.
I've used metaprogramming in enough languages and contexts to recognize it as solving a broad class of problems in a general way, but you still want to pick your moments because it isn't free. Similarly, garbage collection is the right choice for many software applications but it isn't free and there are contexts in which garbage collection introduces far more complexity than is justified by the benefits.
Recognizing these situations and being able to take advantage of them is a market opportunity.
The big wins are in niches that involve a lot of domain-specific knowledge. The two best examples that I was personally involved with were the NASA Deep Space One Remote Agent and the Meta chip design tool from Barefoot Networks (acquired by Intel in 2019). In the former case, an attempt was made to do the implementation in C++, which failed outright. In the latter case you can do a pretty direct apples-to-apples comparison of the design cycle time relative to off-the-shelf design tools. Meta lets you iterate in minutes what would take hours using standard tools. (To be fair, Meta does not do everything that the standard tools do, and before you can tape-out you have to do a few iterations on standard place-and-route and timing verification. But it's still a huge win over just using those for the entire design.)
I have experience with EUROPA, which has gone on to have a long life independent of RAX (e.g. I believe it's still used via SACE for the ISS solar arrays). EUROPA is C++, not Lisp, even though Lisp isn't an unusual choice in AI planning.
I don't doubt that RAX used Lisp, but they replaced it with C++ within a few years, somewhere between HSTS and the second generation of EUROPA. The C++ versions are the ones that have been in 'production', so to speak, for a couple decades now. The early 2000s Mars Exploration Rovers might've been on an old enough version to still be running Lisp, though.
Well, I would definitely be interested in these examples. I occasionally write Lisp (admittedly, Emacs Lisp rather than Common Lisp) and while I appreciate having macros and other metaprogramming tools at my fingertips, I've never encountered a situation in which their use was critically important. I can always replicated the thing I wanted to do in Python with a bit of boilerplate; if I had to choose, I would certainly take Python's huge ecosystem over Lisp's metaprogramming. Frankly, I don't think there have been any language silver bullets after structured programming and garbage collection. So I'm very skeptical of the claims of extreme Lisp productivity. I'm open to being convinced otherwise though.