Note, the power of the macro system is not equal to its general good.
The C macro system was already too powerful, and one of the things that hurts languages is a preprocessor of virtually any kind. The badness of the C preprocessor starts much earlier with things like include files, and the fact that you can't statically analyze the code in any reasonable way.
The benefit from the preprocessor is absurdly small. With that said, I think the benefit of the Lisp preprocessor is pretty small too. C would be harmed more by its omission due its compilation model, but as we've seen with languanges in its family, like C#, its unnecessary.
In a decade or less, the Lispers will see this light too.
The C preprocessor can't do metaprogramming, by definition - there's no way to take apart and examine the arguments passed to macros.
The reason you can't statically analyze C code that uses macros has to do with the preprocessor's broken design - it is a separate stage from the compiler, and there's no way for the two to communicate.
All this actually means that Lisp macros can be used for static analysis (via macroexpand), in a user-extensible way, without needing to add extensions to the compiler or stages to the compilation process.
I don't understand why you cite C# as a good example of something that doesn't have code preprocessors. Visual Studio has T4 (Text Template Transformation Toolkit), which is a dumb joke.
First, the use case for T4 is fundamentally different than language macros. You would't use Lisp macros or C macros for what you'd do with T4. For example, I've seen T4 used to do some elaborate documentation generation.
But back to the point, I partially agree with what you say about C, but I think you misunderstand where the crux of the problem. It's not that they're separate stages -- I believe they are in most Lisp implementations (but I may be wrong), but rather that you can't do macro expansion w/o the context of the full program, i.e., within the context of the build system -- yet the effect is global.
"First, the use case for T4 is fundamentally different than language macros. You would't use Lisp macros or C macros for what you'd do with T4. For example, I've seen T4 used to do some elaborate documentation generation."
I don't think you know anything about the use-cases for Lisp macros. It's a standard practice to use them to generate documents (most CL HTML generating libraries are just sets of macros), configuration information (SQL schemas, XML) for other systems, and programs in other languages (for example, this is how Parenscript compiles Lisp to JavaScript).
"rather that you can't do macro expansion w/o the context of the full program, i.e., within the context of the build system -- yet the effect is global."
* don't think you know anything about the use-cases for Lisp macros. It's a standard practice to use them to generate documents (most CL HTML generating libraries are just sets of macros), configuration information (SQL schemas, XML) for other systems, and programs in other languages (for example, this is how Parenscript compiles Lisp to JavaScript).*
Maybe I don't know the Lisp uses cases for writing conceptual docs, but I must say that this sounds horrible. But clearly each to their own... when all you have is a hammer. I'd much rather use just about any templating system (and there are many) than any language (or preprocessing system) that I've ever encountered.
I don't understand this.
In C, the only way to know the impact of macros on code is to build it (preprocess it). And macros have global effect on the code, so you don't know, when looking at any piece of code in C, what it actually does, unless you look at it preprocessed -- for the most part. This means pulling in the build system, since obviously you don't know what is being pulled from where otherwise.
"I think the benefit of the Lisp preprocessor is pretty small too"
Lisp doesn't have a preprocessor, Lisp is a preprocessor.
You can program with it, but thinking about it as a programming language (ASCII or Unicode Strings to be Interpreted) is at least partially wrong.
"In a decade or less, the Lispers will see this light too."
It hasn't happened in the last half century, so how much do you want to bet on it?
Lisp doesn't have a preprocessor, Lisp is a preprocessor. You can program with it, but thinking about it as a programming language (ASCII or Unicode Strings to be Interpreted) is at least partially wrong.
This would be news to McCarthy. McCarthy on several occassions, including the History of Lisp, refers to Lisp as a programming language. I've never heard him say Lisp is a preprocesser. I certainly am no expert compared to McCarthy. Maybe you are?
It hasn't happened in the last half century, so how much do you want to bet on it?
It's importance has ebbed and flowed over time. This is natural of language features. It was a decade ago where all the rage was template metapgrogramming in C++. Now, outside of a few library writers, most people don't touch it. This isn't to say it won't be popular again some time in the future.
I should probably ammend my statement in that light and say that we'll likely see a major ebb in the coming years.
Are you talking about the importance of Lisp macros here? That's what you appear to be doing. I don't see a decline in enthusiasm for use of Lisp macros among Lisp programmers.
"It's importance has ebbed and flowed over time. This is natural of language features. It was a decade ago where all the rage was template metapgrogramming in C++. Now, outside of a few library writers, most people don't touch it."
The reason that few touch template metaprogramming in C++ is because it is a painful, limited, and in pretty much every way sucky experience. Many of the simplest tasks you wish to perform are either nearly impossible, or often drastically increase the amount of code you have to write. Many of us tried (present company included) and moved on because we figured out what a waste of time it was.
Lisp macros, on the other hand, are a very powerful tool that can reduce the amount of code you have to write (and provide new abstractions to do useful things with, and probably at least a dozen other things I haven't mentioned). They are not perfect, and they can be abused, but they shouldn't even be mentioned with C++ templates in the same breath (unless you are talking about how much better Lisp macros are :-D ).
I see your ammended statement/prediction and raise you with this one: I predict that Lisp macros will continue to be at least as important as they have ever been (at least to Lisp programmers) in the next ten years.
> The badness of the C preprocessor starts much earlier with things like include files, and the fact that you can't statically analyze the code in any reasonable way.
Static analysis hardness or easyness has _nothing_ to do with preprocessing, because static analyzer can preprocess code in a very same way compiler will so both will analyze the very same stream of tokens.
> The benefit from the preprocessor is absurdly small.
It took five years for LINQ to be added to C# because it was too painful for anyone other than Microsoft to add any syntax to the language. Macros would have fixed that.
Lambdas, type inference, extension methods, etc... all good stuff. LINQ itself they could have skipped. And if macros help accelerate more stuff like the syntax in LINQ -- well I think that proves my point.
The C macro system was already too powerful, and one of the things that hurts languages is a preprocessor of virtually any kind. The badness of the C preprocessor starts much earlier with things like include files, and the fact that you can't statically analyze the code in any reasonable way.
The benefit from the preprocessor is absurdly small. With that said, I think the benefit of the Lisp preprocessor is pretty small too. C would be harmed more by its omission due its compilation model, but as we've seen with languanges in its family, like C#, its unnecessary.
In a decade or less, the Lispers will see this light too.