> We crawl this document as an AST, and actually transform it into four formats...The key to getting sensible output is really just having total control of the input format. It's still hard work, but when you can encode the intent of each example into a custom input format, converting that to another format turns out so much better than trying to write something that can convert arbitrary jQuery to React or something.
One tends to think of parsing/compiling as a fairly esoteric skillset that a relatively small number of programmers ever actually need at their jobs, but it always makes me glad to see examples like this where a little DSL was a genuine boon to productivity in a real product. Once you know how to do it, you start to notice more and more little opportunities where it could be useful.
So true. The thing that put me on the path of discovering parsers was a tweet that said something to the effect of “if you want any relatively complex regex, what you actually need is a recursive descent parser”.
Now as a self thought dev I haven’t really been shown in school what those were and always thought of them as some wizardly magic that only compiler writers dabble in.
I once worked on a product where the idea was the end user would write a lot of custom regexes to parse the output of commands on network devices. The system was written in such a way that the regexes were nested and looped depending on the depth of the output. The overall result is you could read and update config on a device based on what output of these regexes.
It seemed bizarre to me that it was considered an appropriate solution. The end users were network engineers so they were _able_ to write the regexes but it seemed obvious to me that providing the user with a higher level parsing library would be much easier. It was something I wanted to POC but never found time in the end.
I have a CS degree but we just never really covered parsing or compiling at all, for whatever reason (well... technically we talked about the formal theory behind grammars a little bit but we never did anything with it, and it hasn't really been useful at all)
But I always dreamed of making a programming language so one of my pandemic projects was to read Crafting Interpreters, and ever since then I've been a little bit obsessed
That's a great point about regexes! I'll have to keep that one in mind
You always have to check yourself and make sure it'll genuinely be helpful, because it's so tempting to do :)
But that said, I had a good usecase of my own a couple months ago where a little s-expression-based DSL was just the ticket to open up some data analysis functionality to internal users without having to prematurely build out a super complex, much more rigid GUI. DSLs are really just small, hyper-flexible user interfaces. Once you frame it that way, there are many applications. (The drawback of course is discoverability/feedback, which GUIs excel at, so you have to balance the two for your usecase)
Someone on here made the comment a little while ago that while that is true for compiler makers, parsing is the much more useful skill for the average programmer. It allows you to approach a new realm of problems that are otherwise off limits.
So while looking back it may seem like a gentle incline, to a lot of us looking forward it is a steep cliff.
One tends to think of parsing/compiling as a fairly esoteric skillset that a relatively small number of programmers ever actually need at their jobs, but it always makes me glad to see examples like this where a little DSL was a genuine boon to productivity in a real product. Once you know how to do it, you start to notice more and more little opportunities where it could be useful.