Writing a Lisp interpreter is an opportunity to write a really beautiful program, do not waste it by stopping at a first version that sort of works. A nice implementation should read almost like a spec of the language, except maybe for the more prosaic parsing part. Look at the Norvig article, or at the implementation of Ian Piumarta for inspiration:
The tokenizer breaks on strings with parens in them. You should be able to split on each character and implement a simple variant on the shunting yard algorithm in only a few more lines of code and get a robust tokenizer.
You might enjoy Lisp in Small Pieces. Is one of my favorite books
The book is in two parts. The first starts from a simple evaluation function and enriches it with multiple name spaces, continuations and side-effects with commented variants, while at the same time the language used to define these features is reduced to a simple lambda-calculus.
The second part focuses more on implementation techniques and discusses precompilation for fast interpretation: threaded code or bytecode; compilation towards C. Some extensions are also described such as dynamic evaluation, reflection, macros and objects.
Great book, although I found it hard parse sometimes (compared to say the later chapters of SICP) which I suspect is because it is translated from french. Either way, it puts so much information in one place, it is incredibly valuable.
If anyone's considering it, now is the time. I've watched it for the last five months. The used price almost always hovers at 80 dollars. It is currently 54 dollars.
I'd buy it myself if it weren't just going to sit on my shelf, unopened, the next five months.
Looks very interesting; I already had a shot at implementing a subset of scheme using F# (https://github.com/fabriceleal/Pang), it would be very interesting to follow this series and check the differences between my and his implementations.
I would also find very interesting to see a step-by-step series on compiling scheme (with macros included) using llvm. I never tried really hard to look for one, though.
Really loved the sense of community that sprouted up around those articles :) I remember everyone following each other's projects on github, and the discussions in the comments on the blog posts. Good memories :)
I should finish my Ada version sometime... Or just start a new project haha
I've written a toy Lisp interpreter that eventually evolved into a toy Lisp compiler. It's a gratifying exercise that I'd recommend for every programmer.
I couldn't agree more. Writing my own interpreter https://github.com/cninja/pEigthP provided many challenges but I think it was one of the most enjoyable "toy projects" that I have done. Also seeing the look on someones face when I say I wrote something to allow me to embed lisp into PHP is priceless.
The existing product was a C code base that had been hacked upon for 16 years (!), with several distinct breaks in continuity. Sufficiently bad that it took hours of investigation to make the slightest change safely, and its performance was increasingly unacceptable, plus it was buggy. I've done a lot of "code archaeology" before and some since, and this is the first time I decided a code base was a total loss with no insurance. So the deadly total rewrite was planned.
The business model was "gated open source", as in paying customers got source, the users were almost without exception system administrators, not programmers, and with one exception who was something of a programmer (at least he did more with the awful code base than any other user) and willing to give it a try, they demanded the "extension" language (actually the complicated business logic) be in Perl ... despite to our knowledge at there were no successful examples of this, whereas Lisp has many, with EMACS and AutoCAD being famous wild successes. (The whole thing couldn't be in Perl because of performance requirements.)
Despite being the only programmer working on it and not seriously experienced with Perl, I got overruled. The CEO, the only manager, was the sort who during this time of decreasing revenue felt compelled to move from Class B to Class A office space (https://en.wikipedia.org/wiki/Office#Grading), and more of it---yet somehow the 3 technical people supporting almost all revenue got allocated only one office, which was tight for 2 people, and I'm one of those who needs quiet for maximum productivity (the other two were sysadmins and technical support, and had to do a lot of the latter over the phone. Whine, whine.)
Basically, only with maximum productivity on my part was this possible to pull off (and I've done this sort of thing before), as the users were years overdue for real technical fixes and progress (long before I arrived).
The board got really upset, the executive director blamed me and they believed her (I after all had been there for less than 2 years and had been able to do only one month of programming in my first year due to Y2K and other sysadmin and tech support demands) ... I had to bail.
It was partly my fault, I was not very productive for several months after my major architectural decision was vetoed and that work was dumped, which happened roughly the time the office move made the company's financials likely terminal, and I should have outright refused the CEO when she demanded I drop everything to help try to sell more of the old (which her fantasy budget demanded) and pre-sell versions of the new (then again I have to wonder if that would have made any difference).
The company died an ugly death, but not before firing in revenge a close friend I'd hired to do system administration and tech support.
Bleah. Bottom lines: avoid recruiting your friends because the company may later go to hell, if you observe the former then don't stay at places that don't respect programmers, can't keep them, and who don't have their eye on the ball of what brings in the cash. None of this was apparent when I accepted their offer, but I'm sure most of us know that story.
I'm currently working on writing one in Haskell using the Scheme48 tutorial [1], though I'd eventually like to replace the evaluator with an F-Algebra[2] so I can learn about that.
As a side note, would any Haskellers be willing to look at the abstraction I did for parsing #-forms? I created an adhoc data structure:
hashLiteralInfo = [
('f', (Nothing, \_ -> Bool False)),
('t', (Nothing, \_ -> Bool True)),
('b', (Just $ many $ oneOf "01", Number . toInteger . fromBase 2)),
('o', (Just $ many $ oneOf "01234567", Number . toInteger . fromBase 8)),
('d', (Just $ many $ oneOf "0123456789", Number . toInteger . fromBase 10)),
('h', (Just $ many $ oneOf "0123456789abcdefABCDEF", Number . toInteger . fromBase 16))
]
And then the parser consumes that:
parseHashLiteral :: Parser LispVal
parseHashLiteral = do
char '#'
c <- oneOf $ map fst hashLiteralInfo
let literalInfo = lookup c hashLiteralInfo
case literalInfo of
Nothing -> fail "Internal parse error: unregistered literal info"
Just (Nothing, f) -> return $ f "unneeded" -- I know there's a more idiomatic way to do that
Just (Just literalParser, f) -> do
digits <- literalParser
return $ f digits
This works and is "abstracted", but doesn't feel idiomatic. Thoughts?
Personally, my approach would be to change the association list so that the values, rather than being pairs of type `(Maybe (Parser String), String -> LispVal)`, are instead just values of type `Parser LispVal`, something like this:
hashLiteralInfo =
[ ('f', pure $ Bool False)
, ('t', pure $ Bool True)
, ('b', fmap (Number . toInteger . fromBase 2) (many $ oneOf "01"))
-- ... and so on
]
The Boolean literal cases are then just parsers which consume no input and always return the same LispVal, instead of being separate, special cases that need to be handled by awkwardly passing a dummy string to a function which ignores its argument.
In particular, with hashLiteralInfo setup as [(Char, Parser LispVal)], the parseHashLiteral function became as simple as:
parseHashLiteral :: Parser LispVal
parseHashLiteral = do
char '#'
c <- oneOf $ map fst hashLiteralInfo
case lookup c hashLiteralInfo of
Just x -> x
Nothing -> fail "Internal parse error: unregistered literal info"
I'm definitely going to go through Edward Kmett's recursion-schemes library which has the zygohistoprepromorphism in it. While I plan to cover topics to make comprehension of the zhppm easier, I don't know that I'll examine it in particular.
Unless I just do for fun, because it's not actually terrifically complex... just sort of a joke name for a relatively simple, abstract idea.
I gave this a try but ended up going with tmhedberg's suggestion because it (as he said) allowed me to stay with lookup. I would use a richer data structure except for the parseHashLiteral function just needs to return a Parser LispVal anyway, so there's nothing "richer" just yet.
However, if I decide to go forward with some of the more advanced things in the tutorial like threading good error messages and such, I'm pretty sure some new data types are in my future.
Also, when you say "underscore for lenses", did you mean that your example would be pretty clean with the Lenses library? I've been looking for an excuse to learn that so if so I'd love to learn more!
This simplifies the type signature, but doesn't avoid the need for `return $ f "unneeded"`, which seems to be the ugly part he was really hoping to eliminate.
Also, now you have to write your own lookup function instead of just using Prelude.lookup.
Is there some reason why every code segment inside not one but two scrolling frames?
And does anyone else dislike web design that (1) limits the text to a column a third of the page wide, and then (2) because this is too small, requires a horizontal scrollbar on code segments?
For an explanation of a small Lisp interpreter (in Python) that uses sane web design, one might consider Norvig's http://norvig.com/lispy.html
Any environments, list-aware reader, closures or just parenthesized prefix notation translated to JS or Haskell?)
btw, not related to OP, those guys who think it is so clever to translate some primitive subset of Scheme into Haskell should appreciate that writing Scheme interpreter in Scheme is a part of CS61A course and it takes one page of much more readable code.
If, for some reason, you wish to write something other than toys, look at Mark Freeley, to realize that serious Scheme system is hard.
You could, I think it would be more interesting to start with a bootstrap system, written in something native, that get's you the minimum you need to run lisp and then rebuild the system in lisp on top of that. Of course, that requires delving into some much more interesting topics like foreign funcitons, compilation, etc.
Who needs a toy lisp? Can't we just lisp for real?
Edit: Ok, ok. Now that the title of submission has been changed from "Toy Lisp Interpreter" to "Little Lisp Interpreter" my quip, having lost context doesn't make me smile anymore either.
http://piumarta.com/software/lysp/lysp-1.1/lysp.c