No need to get all worked up. As I said, I'm not idiomatically against AST manipulation at all. Just that I don't have a big issue with C preprocessor, which is a good solution for the simple cases. It doesn't improve the situation if you replace ## by programmatic access -- it's a little more involved but on the upside you can maybe expect slightly better error messages in case of wrong usage.
In the end, there are zero problems with my approach here, either. And the CPP doesn't encourage you to get too fancy with metaprogramming. Metaprogramming is a problem in its own because it's hard to debug. I've heard more than one horror story about unintelligible LISP macros...
Note that I am going to experiment with direct access to the internal compiler data structures for my own experimental programming language as well. But you need to realize that this approach has an awful lot more complexity. You need to offer a stable API which is a serious dependency. You need to offer a different API for each little use case (instead of just a token stream processor). If you're serious like Rust you also need to make sure that the user doesn't go all crazy using the API. Finally, it's simply not true that you need to understand less about parsing (the syntax) with an AST macro approach. The AST is all about the syntax, after all.
In the end, there are zero problems with my approach here, either. And the CPP doesn't encourage you to get too fancy with metaprogramming. Metaprogramming is a problem in its own because it's hard to debug. I've heard more than one horror story about unintelligible LISP macros...
Note that I am going to experiment with direct access to the internal compiler data structures for my own experimental programming language as well. But you need to realize that this approach has an awful lot more complexity. You need to offer a stable API which is a serious dependency. You need to offer a different API for each little use case (instead of just a token stream processor). If you're serious like Rust you also need to make sure that the user doesn't go all crazy using the API. Finally, it's simply not true that you need to understand less about parsing (the syntax) with an AST macro approach. The AST is all about the syntax, after all.