For any struggling to understand Ruby's metaprogramming, the rather short and succinct Metaprogramming Ruby is an excellent read. There's a method to the madness of define_method, method_missing, the various eval methods, the difference between Blocks, Procs, and Lambdas. Most of it revolves around manipulating scopes.
It also shows how you can iterate from the ugliest, most powerful, most dangerous eval method by pulling out the functionality you need into less-powerful, safer manipulations. It's all Ruby code until it's actually executed, so it can all be manipulated at runtime.
Seconding. The book revolutionized my understanding of Ruby, and inspired me to create a whole visual novel framework: https://github.com/thirdtruck/rubyai.
While I agree with all of the benefits of DSL writing, I kind of feel like DSLs in Ruby are a bit silly now. There is always a thin line between "lots of function abstraction" and DSL (e.g. a shallow embedding is pretty tough to distinguish from... well, just a library) and that line is even thinner in the fast-and-loose world of Ruby DSLs.
It's a vague enough question to lead to nothing but religious wars, but I really want to push people who are interested in (e)DSLs to consider their implementation in typed languages. Haskell is, of course, a favorite of mine, but you can achieve similar things in Scala and the ML family. You can even go much further using something like Idris, mostly by using its stronger dependently typed system.
Haskell is not very suitable for the efficient and straightforward eDSL implementation. Template Haskell is a nice thing, but it is staged (cannot use the template definition from the same module) and enforces explicit template syntax. And the usual Haskell DSLs are in fact all just ad hoc interpreters, missing all the benefits of having an underlying compiled runtime environment.
And typing is not a big deal, once you have a proper metaprogramming, you can easily add typing on top of your underlying language.
You can take advantage of the Haskell compiler with a shallow embedding, you can do your own compilation steps with a deep embedding, you can do partial evaluation to speed up compilation.
You can introduce your own type system, of course, but it's really nice to have most of that infrastructure pre-built for you (otherwise why embed?).
The only reasonable ways of implementing fully compiled DSLs in Haskell are standalone DSL compilers (e.g., as in Happy) or Template Haskell (restricted syntax, stages, limited reflection).
Both ways are not as convenient as with the more metaprogramming-oriented languages, plus the lack of reflection limits the potential for integrating DSL into the host language and the other DSLs. Yet, TH is really nice. I cannot stand languages without any metaprogramming capabilities at all, and TH just ticks the box for me.
Compilation is much simpler than interpretation, especially an ad hoc one, in a Haskell or Ruby idiomatic style.
Compiled DSL implementations are much more declarative - they simply define how a DSL is mapped to the other DSLs or a host language.
Compiled DSLs are more efficient - you don't want an interpreted regexp engine, don't you?
Compiled DSLs are IDE- and static-analysis-friendly, at no additional cost.
Compiled DSLs are more modular and reusable - by sharing the same runtime environment, they can be easily mixed together, their fine grained properties can be picked individually and mixed into new DSLs.
Having said that, I see no single advantage of the interpreted DSLs.
If you can provide me with any evidence that interpretation can be more efficient than a static compilation outside of the marginal cases, I'd be very grateful.
I did a lot of research in tracing JITs, runtime partial specialisation and all the related stuff, but so far there is no practical and useful implementation of any of such techniques which could beat a mere straightforward static translation.
I never claimed it could be more efficient. I'm completely in agreement that compiled DSLs are more capable. I'm even a big fan of eDSLs which produce compiled output. I don't find completely static, compiled exterior DSLs to be as frequently useful, however, due to the massive increase in maintenance cost---this holds especially true when you have a sufficiently expressive type system to embed the type system of your eDSL into.
Then I did not understand your reference to the 3rd point.
I'm not talking about fully standalone compiled DSLs (totally agree they're usually unmaintainable) - I was referring to the metaprogramming-based compiled eDSLs, implemented as macros on top of a host language (or a range of host languages). This way it's really easy to mix traits into your DSL design, including various type systems, not even necessarily directly compatible with the host language type system.
I found it very productive to have Prolog (or anything comparable) as one of the host languages, it's trivial then to map most of the practical type systems to it.
Wow. That was an amazing article. I have been doing Rails/Ruby for years and never understood any of that. Thanks for putting that together, please consider doing more tutorials for other interesting things going on in ZenPayroll.
Programmer with a couple of decades' experience here, but no ruby. I have no idea what half the tokens in the example mean, or what changes would be valid beyond editing the four obvious values.
A good read for anyone just starting out with some of the more intricate parts of Ruby.
I often think Ruby is unsurpassed for writing readable, English-like DSLs - perhaps because of the combination of the convenient lassitude of Ruby's syntax (not requiring brackets - in many cases - or semi-colons) and the dangerous power of its metaprogramming abilities. The only other language that perhaps comes close and immediately springs to mind, before I've had my first coffee of the day, is (appropriately) CoffeeScript. Any other takers?
Well, obviously on the type safe DSL front nothing compares to Scala or Haskell.
Groovy's MOP was a bit buggy/lacking when I was using it (this is as of 3 years ago mind you, not sure how Groovy MOP has evolved since then) but I recall Gradle and Spock being impressive DSLs, very natural/readable.
> Spock being impressive DSLs, very natural/readable
The Spock syntax isn't a DSL, it hacks the ossified restrictions of the Antlr 2 based Groovy syntax to pull off tricks like overloading the | operator to give the appearance of tables, and the break/continue labels via an AST plugin to replace function calls with sections, as in this oft-quoted example from their website:
expect:
Math.max(a, b) == c
where:
a | b | c
1 | 3 | 3
7 | 4 | 4
0 | 0 | 0
Languages with expressive type systems, Haskell likely the most mainstream, allow for deep embedding of an DSL, such that the type system of your DSL can be embedded into the parent language.
IMHO absence of static analysis is a huge downside of using an interpreted language. Having type safety in an embedded DSL can only be dreampt of in a language like ruby.
on the jvm i would say groovy is probably one of the better languages for dsl's. Gradle is a dsl and groovy has inbuilt Builder classes to help you in the creation of dsl's
Clojure being a lisp is extremely powerful in this regard as well, but if you are programming in any lisp I would expect a fair amount of meta programming.
I wrote it because I used an invoice app, that went bust and I needed something really simple to send out invoices and I was in complete control of.
Also I didn't want to use a database so just used Ruby DSL meta files that described my invoice.
Admittedly it probably needs a bit more work... but it was fun to do.
It also shows how you can iterate from the ugliest, most powerful, most dangerous eval method by pulling out the functionality you need into less-powerful, safer manipulations. It's all Ruby code until it's actually executed, so it can all be manipulated at runtime.
http://www.amazon.com/Metaprogramming-Ruby-Program-Like-Pros...