Urn effectively tries to be it's own language which sits on top of the Lua runtime. I've tried to take the nicest bits of Lua and combine it with macros and a little more compile-time checking. Though I'm not entirely sure how successful I've been...
Urn was is very much an experimental project for me: part of me wanted to see how much I could do with as little syntax. Consequently Urn has comparatively few "special forms", but meaning it generates less idiomatic Lua.
Fennel is much more an alternative syntax to Lua. Most Lua constructs have a corresponding Fennel form, meaning the generated code looks more "like Lua". Lua interop is also much nicer: Urn requires you to declare all variables, so you have to jump through hoops to use external libraries. With Fennel you can just go ahead like you would with Lua.
I've toyed around with ideas related to your project. Such as: What if... you had a Lisp that instead of being based around the classic linked list, was based around the Lua table instead?
I've briefly tried to come up with formulations for how the code itself should be represented as a series of nested tables.
And perhaps have a series of nested tables for all variables in the runtime as well (including local ones), though Lua sort of does this now.
All that would increase ease the introspection, creation of macros, and other fun stuff. I hadn't started deciding on a syntax or anything either.
Urn seems to be greater in scope and code. Fennel is intentionally small and a single file that can be dragged into any project.
Fennel does not try to provide Lisp idioms or abstractions unless they have a direct mapping in Lua. For example, there are no linked lists (use tables), and no continuations (use coroutines). Operators are implemented as special forms rather than function literals so that they can be easily implemented efficiently.
Fennel also includes abstractions that might not make sense in a normal lisp but help make interfacing with Lua code easier. For example, symbols that have a dot in them, like "table.insert", have the correct interpretation and can be used as normal symbols. Method calls are supported such via the `(:)` form; `(: "hello" :find "llo")` compiles to `("hello"):find("llo")`.
Fennel takes this approach because Lua is almost already a Lisp, so I felt it was best to add the minimum abstraction necessary. Also, I prefer simple and transparent tools.
Urn may provide more comfortable and "Lispy" abstractions, however, because it takes more control of the environment.
It also seems to have pretty great error messages compared to Fennel, and a good website. There are definitely some features in Urn that aren't
(yet) in Fennel, like built in pattern matching, although Fennel does do destructuring.
I would like to know too, and to Lumen also mentioned in a thread here. I may try both, and see what is closer to Lisp, Clojure or a combination of both. Thanks for pointing out Fennel.
I have never asked this before, but Lua seems to have a couple of different languages implemented in itself, almost making me think of how JavaScript has a similar offering. Has anyone compared JavaScript and Lua in the language building regard? I'm curious to see what the most commonly used JS / Lua features are when designing a new language on top of those two languages. I know some languages have features that help you to build a new language (or it seemed to be like that) I think Java / C# do but I don't remember the names of the classes for that stuff, last I saw it was many years ago. Also of course I know Lisp lets you get there too, if I were to try to create my own language I'd try building my own lisp as a first project, and then go from there to try and make something different.
If you are interested in how to implement languages I've not yet come across a better introduction than the classic "Structure and interpretation of computer languages". The 'build your own scheme' section is transferable to any other language.
https://mitpress.mit.edu/sicp/full-text/book/book.html
On the other hand, I've been building languages for a few years, and I never got anything useful out of that book. If you open it and feel dumb, don't worry, it's not just you.
MIT and especially books from MIT are held on a pedestal. If you can't understand them or don't feel like pursuing them, just find a different book.
Everyone to his own taste, but I think that is just short selling the book. I had no problems in starting to convert the lisp evaluator described in the book to C++ after only a few years of programming experience and I'm not an ace programmer. It was not straightforward, but it was not intimidating either. I did it for fun, not as a compulsory exercise.
Specifically, Chapter 4 (Metalinguistic abstraction)[1], starting on 4.1 [2]
Sure, you need to figure out that you need to tokenize strings to symbols, and do recursive descent parse stage or such to push the token/symbol list into hierarchical lists. But that's precisely the sort of thing that I feel figuring out has made me a better programmer. The key was that I knew the solutions had to be simple, so I sought simple solutions.
Yes, I agree, the book could have verbatim explanation - "hey, after you know some Scheme go to chapter 4, where we give you a damn good literate programming based exposition of a scheme interpreter." - and since it doesn't, I do think your critique does have pedagogical merit. I remember leafing through the book and marking stuff up, trying to figure it out, until I realized it was standing there in front of my eyes the whole time. My paper copy is quite dog-eared by now.
> Everyone to his own taste, but I think that is just short selling the book. I had no problems in starting to convert the lisp evaluator described in the book to C++ after only a few years of programming experience and I'm not an ace programmer.
@sillysaurus3 just made that book more accessible to everybody by giving them a guilt-free exit, based on experienced results, and your first instinct is to pull a quote out of context and explain why their personal experience is “wrong”?
I'm sorry. I was not trying to be confrontational. Sillysaurus said he did not see much merit in perusing the book after professing some experience in language implementation, and I felt an answer based on personal learning was in order as a counterpoint. Good Books, that actually teach something and mean something are exceedingly rare... this quality, is of course often quite personal.
The concept of 'guilt' when abandoning a book is completely foreign to me. I presume it is a facet of poor self esteem? This is quite useless as a concept and people need to find their own intellectual feet. Not all need to dig the same things.
If some weird book some unknown dudes in the internets glorify feel useless to me I just dump the book without a second thought. I don't have the time to become an expert in every subject, so I should only investigate those concepts that arouse a personal feeling of beauty. Doing brainy things "just because you imagine it makes you look smart" is counterproductive.
> I was not trying to be confrontational. Sillysaurus said he did not see much merit in perusing the book after professing some experience in language implementation, and I felt an answer based on personal learning was in order as a counterpoint. Good Books, that actually teach something and mean something are exceedingly rare... this quality, is of course often quite personal.
Thanks for the thoughtful response. I think sillysaurus was trying to ease anxiety for people (like sillysaurus) who may not get something out of an otherwise highly regarded book. Sillysaurus backed up the statement by saying they had successfully implemented DSLs many times, despite not getting anything from this book. I think this sentiment expressed by sillysaurus is valuable, and in a domain that is often elitist and somewhat hostile, I thought sillysaurus’ position was worth defending for the sake of anybody who might be approaching this with any apprehension. I was advocating for people that might need some encouragement, which is what I saw in sillysaurus’ comment, and that I thought you were tearing down. I appreciate you intended no ill will though.
I got a lot out of "Essentials of Programming Languages" (2nd edition, at the time) by Friedman, Wand and Haynes.
It has a great quote from Hal Abelson in the foreword, too:
> Perhaps the whole distinction between program and programming language is a misleading idea, and future programmers will see themselves not as writing programs in particular, but as creating new languages for each new application.
Mainly the key is to start tinkering, rather than jump from book to book. Just start with a very informal v1 and start hacking stuff together. When you run into a design dead-end, you'll know it because it will become increasingly hard to make forward progress.
And that's the tricky situation. When you run into a design dead end, what do you do?
Books! :) And that's where SICP might end up useful.
Or think about the problem really hard, and write down the answer. I've occasionally pulled that off.
Oh, I thought of one other suggestion: linkers and loaders. It's another classic. And unlike all the other references, there's no Lisp.
I second HtDP for both its material and fact that it has batteries-including, GUI-based tooling to go with it in form of Racket Scheme. Once you know it, you can approach the other resources much better. You can also use your familiarity with macros/DSL's to prototype features you find interesting from other languages in Scheme for your own programs. You can experiment with different styles since the LISP's are among the easiest to do multi-paradigm languages in. Lastly, you can even embed low-level, imperative constructs in DSL's to get benefits of LISP when doing efficient, unsafe coding much like Galois does for Haskell and C with their Ivory language.
sillysaurus3 is right about the MIT books being held on pedestals. It's not just them but happens with lots of "elite" colleges. There's usually much better books for learning. Now, where I disagree is that SICP is a good book with plenty of value in it for people that know Scheme. I loved reading all the sections on deriving interpreters. It was so clean-looking compared to what I did in imperative languages. It's just best to learn programming and Scheme elsewhere first.
I think the thing that made it all click for me was learning how Pratt parsers work, easy to get something up and running in a jiffy while just playing around -- especially if you play around in python. Tons of blog posts on the subject these days.
My current parser fetish is Earley parsers but they're a bit more complicated to get from (possibly multiple) parse trees to an AST you can do something with, still trying to grok how that all works. Probably doesn't help I've been playing with a C++ grammar in my experiments.
Christian Queinniec’s “Lisp in Small Pieces” (LiSP) has always intrigued me: it contains a sequence of interpreters and compilers for scheme-like languages that demonstrate a variety of ways of implementing languages.
Lua is fantastic for implementing languages. I did an implementation of Golang in it (https://github.com/gijit/gi). With metatables one can implement any kind of object/inheritance/interface scheme. Full builtin asymmetric coroutines, and a massively performant, ABI compatible backend (a just-in-time trace compiler) in LuaJIT are amazing tools.
That's an interesting insight. I think the reason has less to do with the languages being good for language development than it is about other language features. JavaScript is the Internet's only frontend language. For many C/C++ programs, users may write plug-ins only in Lua. The common aspect is that the users are forced into choosing the language for external reasons. Many of them see flaws in JavaScript and/or Lua and would like to write in another language instead. That's how I see transpilers coming into existence for these languages.
I can't deny it! Though Urn was much more a "I want to do something with Lua" project, than something targeting CC. That being said, I know people have got the compiler running on various other fantasy consoles.
That, combined with Guile-Emacs would be super sweet.
Not that I have anything against Lisp, but an emacs-ish text editor with Lua as an extension language would be really cool. I know the Zile project tried something like that, but I have no clue how far they have gotten.
Also, there is a huge body of code written in elisp that has been around for ages and got properly debugged along the way. Replacing that is not a very realistic task.
i have recently become interested in lua, especially the fun looking love framework [https://love2d.org/]. but one thing i have had trouble understanding is what lua is particularly useful for and/or used for. can someone explain what type of use cases are up lua's alley? thanks.
One of the most popular and compelling real world uses of Lua is for embedded scripting. It's easy to embed, it's easy to use, for a scripting language it's blazing fast.
I'm sure Lua has other sweet spots, but that's the one I'm most familiar with.
I've mostly experienced it through World of Warcraft -- it's embedded there as the language used to write the game's user interface[1]. Then it's exposed to third-party authors as an API, so they can write addons for the game.
It's also embedded in Wikipedia[2]. Complex templates can be written with Lua modules, rather than relying on wikitext.
I was introduced to Lua via writing a Wireshark plugin for packet verification. It was hard in the beginning but dove right into working with a C backend. I'd say that's an approachable side project if you want to explore Lua.
It's basically the simplest non-painful-to-work-with programming language that doesn't have a lispy syntax. This makes it very useful for app scripting and programming language experimentation. (And other use cases where you want a dead simple language, but are for some reason averse to parentheses)
Also, Lua's C API for interfacing Lua and C is very pleasant to work with. I did not really get why Lua was so popular until I used it as a scripting/extension language in a small toy project. That was an eye-opener!
I made fedcrawl.com, the entire backend runs on Lua/torch. This involves scraping federal contract sites, cleaning text, and ultimately indexing everything on elasticsearch. It's really just a few Lua scripts running as services via systemd and passing messages with redis. I love it for the speed and how simple the code looks.
At work, I use Lua and LPeg [1] to process SIP messages. It was easy to embed, the coroutine support makes writing an event driven daemon easy (the Lua code is imperative despite being event driven) and LPeg makes parsing text super easy. It's also proven "fast enough" that no real attempt at optimization has been done.
My exposure to Lua is limited around writing Hammerspoon scripts. Lua is nice but I really wanted to write those scripts in a Lispy language, maybe something Clojurescque. Has anyone done Hammerspoon either in Urn or Fennel? Which one should I try? Would that even work?
https://github.com/bakpakin/Fennel