I went through this a while ago (in Prolog). It was interesting, but with a lot of room for improvement. My criticisms as I remember them:
- Error messages about output being different from the expected use some sort of very explicit printing mode with lots of quotes and escaping. A message saying that one got "\\\'\\\'" where "\'\\\\'" was expected is needlessly hard to parse when the actual difference is \'\' vs. '\\' (or whatever).
- Many parts are marked optional, but the test script complains if you don't implement them, and later sections assume that they are implemented, so you have to treat them as compulsory anyway.
- Some things regarding environment updates were contradictory or at least not explained properly at first, with some very relevant information only coming later. I don't remember the specifics, but the problem was roughly that I set up my data structures thinking that everything could be implemented in a pure way, but at some point it became clear that you either need some other data structure or must use destructive modifications.
- There are several reasons for the weird way that the output and expected messages are printed. One of the main reasons is that in order to enable more flexible test case results, the expected output is a regex rather than a plain string, while the output is just a plain string. There is probably a way this could be made clearer. I'll add it to my TODO list to look at.
- Do you have some specific examples? The test cases are marked as either deferrable or optional. You shouldn't ever have to implement optional (and if so that's a bug in the tests or the guide). I think the deferrable items are marked pretty clearly in the guide where they become mandatory in later steps. If it's not clear, then that's a bug.
- This is one of the tensions that exists with trying to make the guide incremental; later features may require re-work of earlier functionality. I do try and minimize that as much as possible although I've found it can really vary depending on the nature of the target language. Note that the primary goal of mal/make-a-lisp is pedagogical (as opposed to say "the easiest way to make your own Lisp"). So sometimes the need to go back and re-work something is in line with that goal.
If you have any concrete guide text or test driver improvements (especially that further the pedagogic goals of mal), I'm always happy to review pull requests! :-)
Thanks for your answer. I probably meant deferrables rather than optionals that were de facto required. Sorry I don't have a concrete example handy, it's been a while.
- Error messages about output being different from the expected use some sort of very explicit printing mode with lots of quotes and escaping. A message saying that one got "\\\'\\\'" where "\'\\\\'" was expected is needlessly hard to parse when the actual difference is \'\' vs. '\\' (or whatever).
- Many parts are marked optional, but the test script complains if you don't implement them, and later sections assume that they are implemented, so you have to treat them as compulsory anyway.
- Some things regarding environment updates were contradictory or at least not explained properly at first, with some very relevant information only coming later. I don't remember the specifics, but the problem was roughly that I set up my data structures thinking that everything could be implemented in a pure way, but at some point it became clear that you either need some other data structure or must use destructive modifications.