Personally, I prefer using paredit because it feels more predictable since you're telling it what to do explicitly. That said, I've heard from a lot of beginners that parinfer helped them get started with Clojure. Even if it's not optimal, I think there's a lot of value in the fact that it's easy to get started with.
I've never been a fan of paredit, but paredit, smartparens, and lispy are all far more powerful and (for me as well) more intuitive. I agree that parinfer deserves credit for being widely available for different text editors and for being fairly easy to pick up. I do think that the method is unnecessarily complicated though. When "paren mode" was still necessary, I think it was a much poorer choice for a newcomer than it is now.
That's why few people are using things like parinfer. This demo might look cool to non-lisp programmers, but in lisp-land, everyone is using semantic editing (in Emacs or otherwise), and not line-based editing.
That's why few people are using things like parinfer.
I don't think this is true. I develop Cursive and I use it, despite being competent at paredit. A ton of my users do too, even experienced ones. It's just easier, and it turns out that a lot of editing is actually line based, even in a lisp. And when it's not, I can still use the paredit commands in parinfer mode, at least with Cursive.
It's referenced in an edit to noctuid's document, but many of those criticisms are no longer valid since smart mode became the de-facto default for parinfer. It's now really intuitive. Parinfer's worst problem now is its difficulty inferring parens when there are multiple expressions on a line (i.e. when there's no indentation to infer from), but it's surprising how infrequently that's actually a problem.
Paredit is really not the closest or best comparison. If you want simplicity, lispy is often simpler than or at least as simple as parinfer. At the same time, it's more powerful than paredit. Even if you never used its special keybindings, I'd argue that lispy is more intuitive and has more useful default keybindings. The indentation-based inference doesn't add anything major as far as I can tell; it seems to just cause unnecessary problems. I think parinfer is more popular in editors besides Emacs because of the lack of alternatives more than anything else.
What percentage of users is that, assuming that you don't make parinfer the default choice in your IDE? What percentage of Cursive users knowingly say: I'm a bit confused by how these parens work (my guess: all lisp newcomers), what percentage goes to preferences to see what they can do about it (my educated guess: a lot, but less than, say, a third), and what percentage of those conciously look for parinfer (I don't know - I'm asking)?
Now, Cursive is used by something like a third of Clojure users. Half of them use Emacs, and in Emacs land there are rather few Clojurians who seek something like parinfer. Even when they do, it's a usual "how we can make this more palatable to outsiders" type of quest rather than "I'm really frustrated by how Emacs does Lisp editing. I'd like something like parinfer"...
I don't know exact numbers, since I don't do any user tracking. But I know of numerous users who are already experienced with Clojure who have switched, myself included. Cursive currently has paredit on by default, but I'm going to switch that to parinfer.
FWIW Emacs users do ask about parinfer in #parinfer on Clojurians because they actually want to use it. There currently isn't a smart mode for Emacs though, and that's what I consider the game changer. I suspect that uptake will be good once that's available.
Parinfer seems popular in the clojure community (even among those who have been writing it for a while) and among people new to lisp. In its favor, parinfer's listed selling point is simplicity and not power. I don't think it does simplicitly perfectly, but I think it does it better than paredit, for example.
I loved using Parinfer when I was writing ClojureScript, but it also made me question the value of Lisp syntax.
If brackets can be inferred from indentation, doesn't this imply that they're extraneous — that indentation only would be sufficient to write many Lisp structures?
One can do that, but most Lisp programmers value the direct "source code is data" nature of Lisp - using s-expressions.
Alternative notations were originally planned for Lisp when it was designed - some books were published using it, but for actual programming it wasn't used too much - especially since most implementations actually a) didn't support it or b) had s-expressions as a default/internally.
Then several attempts to modernize Lisp with a different syntax mostly failed to get traction or failed big (like the Lisp 2 effort).
Then some efforts failed to provide more than one syntax and also failed to appeal to people (Dylan).
What we saw is a bunch of languages derived from Lisp with different syntax (ML, Logo, ...) and a bunch of Lisps for mathematics with different syntax (Reduce, Macsyma, ...).
But the core Lisp programmers were never willing to give up the s-expression-based syntax. It has some clear advantages when manipulating code, especially in interactive programming.
Thanks for these comments. I didn't realise alternative notations were planned and never took off. (Dylan in particular looks really interesting.)
I can see the value of homoiconicity / code as data from the perspective of someone writing a parser.
I've asked what writing code as s-expressions gets you as the end-user of a language before. People usually say, “macros” or “it's easier to manipulate code”. But macros exist in non-homoiconic languages, like Nim and Elixir. And in my short time using ClojureScript I'm not sure what manipulations Lisp syntax gave me that I don't already have with other languages in a modern text editor.
I'd love to see examples of accomplished Lisp users on YouTube or elsewhere, to see what those code manipulations look like.
It's true that macros exist in languages like Nim and Elixir, even C. They either use something more akin to string templates (expanding a macro fills in the holes) or a DSL for manipulating the AST (something new to learn to write code that writes code).
One reason Lispers find macros so beautiful is the simplicity of writing macros using the exact same list processing functions. The code-is-data thing is reinforced through the fact that the AST is one-to-one with the syntax.
> I didn't realise alternative notations were planned and never took off.
See for example this program from McCarthy, 1960. On top is my Common Lisp translation and below you find the original code. The original had to be hand-translated into s-expression syntax to run it in Lisp - in 1960.
> But macros exist in non-homoiconic languages, like Nim and Elixir.
Lots of languages have some kind of macros: C has textual transformations and Elixir has AST transformations.
Lisp macros are different, since they are not AST transformations but token tree transformations. That means Lisp accepts arbitrary token trees (give that the first item is a macro identifier) and one can write programs to transform them.
Languages which support macros for AST transformations, need to have the code parsed into an AST first. That means the expressions it can transform need to be already parsed into an AST - and the language accepted by the parser limits the expressions - unless one can add syntax changes to the parser, too.
ASTs are special purpose data structures, while s-expressions are general purpose data structures and thus are not limited to expressing programs in a particular syntax and are not limited to express something which looks like a program.
Lisp offers a bunch of macros: macros for transforming Lisp code, macros for transforming symbols, compiler macros for optimizing code, macros for transforming s-expressions, ... Additionally the macro has access to a full Lisp - which means it can do arbitrary computations: primitive transformations, complex language transpiling, full parsing&transformation, doing side-effects in the development environment, running at runtime in interpreters, ...
That means for me as a programmer I can automate all kinds of code manipulation. There are many simple and many complex examples.
Another advantage is that the language core can be reduced to a minimum set of syntactic forms and every other syntax can be built on top of it - and the developer uses the same mechanism as the language core: macros. This makes macros very pervasive - a programmer will use a lot of macros and may also program macros.
One of the disadvantages is that macros also cost us: the code we see can be very different from the code which executes and there is an additional programming paradigm to understand: using syntactic abstractions with macros. One can learn some Lisp without touching macros (see for example SICP, which does not use macros), but in real world programs one might see a lot of macros.
An example for a typical use of macros is CLOS, the Common Lisp Object System. It has a three or four - layer architecture:
layer 0 : meta object layer for CLOS - here CLOS is implemented in itself
layer 1 : object layer for CLOS - here CLOS deals with simple representations of itself
layer 2 : functional layer for CLOS - here have functions operating on CLOS objects
layer 3 : macro layer for the CLOS programmer - here we have macros providing ways to configure CLOS programs
Take for example defining a CLOS class:
0) on layer 0 CLOS has CLOS functionality to represent and create instances of classes and metaclasses, he were can also write extensions to new kinds of metaclasses
1) on layer 1 CLOS classes are objects: which have slots which are objects and which have superclasses which are objects
2) on layer 2 CLOS provides functions to create all that stuff: classes, slot objects, ...
3) on layer 3 CLOS provides the DEFCLASS macro which at macro expansion time - which typically will be triggered by compilation - can do all the transformations and can interface to the development environment at compilation time.
The programmer will typically use layer 3 to define a class using the macro DEFCLASS. For more advanced use one could use the function ENSURE-CLASS
Common Lisp originally in 1984 did not have any syntax for defining classes. The first CLOS implementation was a Lisp program which implemented all the layers above and provided the necessary syntax to define the new language objects: generic functions, methods, classes and some others. So it was mostly a user-level program which added a full complex object system - with only little code for implementation specific stuff.
Thus the DEFCLASS macro could introduce any syntax it wanted as a user level program - since it is not constrained by a fronted parser. So the CLOS developers came up with something that was convenient for them to define classes. This could mean that it does something trivial by providing a different order to define things, make things shorter to write or that it does something more complex by actually checking the provided class configuration and doing some code transformations.
Thank you so much for this thoughtful reply — I enjoyed the code samples and the details on Lisp's macro system, and I can see how its approach gives you more control to invent on top of the language. It's given me more to read and think about.
dylan maintainer (bruce something) had lots of valuable insights about this
I think I keep lisp syntax~ at heart because its nature made me learn and think recursively which is something of a rare value (and I'm not saying this as a cult, it really made me solve problems I couldn't in ways I was confident it would either work or be clear that my perspective was wrong, very very nice feeling). Another example is that I still prefer car/cdr even though they're laughably cryptic .. but they embody the whole culture (and have a tiny symmetry to it).
> If brackets can be inferred from indentation, doesn't this imply that they're extraneous.
Yes; this deduction is valid.
However, can brackets be inferred from indentation.
Let's start with a trivial case:
[indent]15
Is this object 15? (15)? Or ((15))? How do we apply indentation to distinguish them?
Do we count the spaces? Indentation is two-space so that 15 preceded by no extra indentation relative to the current level is just 15, then 15 preceded by two space is (15), and so on?
And is that really going to be readable?
What if there is whitespace?
material material
;; .. vertical break
15 ;; this is (15) but could be 15 if I don't see the preceding material.
There is ambiguity in indentation. If I see this:
(1 2 3
4 5 6
where 4 5 6 is the last line in my edit window, I know that this list is not well-formed: zero or more items must follow, and then a closing parenthesis. If indentation is used, I have no idea: I could be looking at the complete thing.
I think the example I typically go to is to ask how to automate cutting/moving text around the editor. Having had YAML blow up on my team enough in the past few months, I've grown quite weary of any inference schemes.
Gets even worse, because as soon as you get into the indentation game, you open up the debate about how many spaces. Such that now some of our YAML files are using two spaces, and some more. Portability of the code is basically nil. Worse, it will look right, but won't mean what someone meant for it to.
There are other concerns beyond just writing code. One huge advantage of s-expressions is that your code is written using literal data structure notation. This allows you to serialize code, and to trivially manipulate it using macros as any other data. This is one of the main features that sets Lisps apart from other languages.
That's exactly the conclusion the Haskell community came to, and how they ended up with optional meaningful-whitespace. It's genius and I wish all other languages gave you the ability to optionally choose between braces or whitespace. It could settle the whole Python vs C syntax war once and for all!
> It's genius and I wish all other languages gave you the ability to optionally choose between braces or whitespace. It could settle the whole Python vs C syntax war once and for all!
I really like Haskell as a language, but the fact it isn't based on s-expression is the biggest issue I have with it. Whitespace, braces, etc. are very minor quibbles: they're basically debates about language as a UI. The problem is, that same language is also the API we must adhere to when manipulating code programatically, e.g. writing interpreters/compilers/documentation generators/code formatters/linters/renderers to HTML or LaTex or whatever/static analysers/verifiers/model checkers/IDEs/refactoring tools/etc.
I think it's telling that a language so heavily focused on language implementation (DSLs, etc.) effectively has a single usable implementation (GHC), and a mountain of dead/niche implementations ( https://wiki.haskell.org/Implementations ).
Imagine we're given the path to a file containing Haskell code, and we want to transform it in some way (e.g. replacing calls to one function with another function). The most basic thing we need to do is parse it, but my work on code analysis tools over the past few years has taught me that we can't even manage that reliably.
We might naively reach for the GHC API, but that requires that we set a whole bunch of configuration options that we may not know (language extensions, package databases, commandline flags, etc. collectively referred to as "dynflags"); if we get those wrong, our program crashes saying `the impossible happened!`. We can't, in general, figure out what these should be; the only real solution is to invoke our program via Cabal, and read in the needed values by emulating the commandline flags of GHC. This solution is useless if we have a string of code, without any associated Cabal project.
We might instead opt for a standalone library, like haskell-src-exts. The problem is, those libraries typically can't parse Haskell code found "in the wild". Language extensions are one problem, but another major blocker is widespread use of the C preprocessor (an unhygenic, unsafe macro system based on string substition; which would be largely unnecessary if Haskell used an easily manipulated format like s-expressions instead).
Note that even GHC can't re-use its own ASTs: the Template Haskell extension provides its own AST representation, entirely separate to that used by the compiler's frontend!
Whilst there are layers on top of Haskell like "liskell", which accept s-expressions and produce Haskell code, they're solving the opposite problem: it's easy to convert from s-expressions, since they're so trivial to manipulate. The hard problem is being able to do anything useful with the mountain of existing code which isn't in a nice s-expression format, other than compile it with (some versions of) GHC, if invoked with the right options from (some version of) Cabal; or maybe Stack; maybe after running Hpack; or who knows what else!
> Note that even GHC can't re-use its own ASTs: the Template Haskell extension provides its own AST representation, entirely separate to that used by the compiler's frontend!
Isn't that about the expression problem, though? How would changing the syntax help?
s-expressions are already (a trivially deserialised encoding of) an AST representation. If the Haskell language used s-expressions, or if the "frontend" language was designed to desugar into s-expressions via a standalone pre-processor, then this representation would be available to any code that wanted to use it. Different programs, or different parts of the same program, might decide to convert it to their own specialised datatypes (like those of GHC's frontend and Template Haskell), but those would essentially be details specific to that program (either as an implementation detail or an API). It would have no real influence on how others choose to process the language.
If we think of the current GHC implementation in this way, we find that the only de facto representations of Haskell code is `ByteString` (or `Lazy.ByteString` or `String` or `Text` or `Lazy.Text`, of course ;) ). Since these are generally unparseable (e.g. via haskell-src-exts and friends), it's hard to do much with them (hence the tendency to delve into GHC's own representations).
Note that for my own work I ended up abandoning Haskell syntax altogether in favour of GHC Core. I use Cabal and GHC to do the parsing (since that seems to be the only way to handle real world code), and a GHC plugin to dump out Core as s-expressions http://chriswarbo.net/git/ast-plugin
That’s the attitude of Python with curly braces. But with lisp the significance of indented structures is up to the implementation (`if` and `let` are the same “syntax” but are conventionally indented differently).
>that indentation only would be sufficient to write many Lisp structures?
Lisp's syntax is just a textual representation of the internal abstract syntax tree, so you could represent it any number of ways: s-expressions, indentations, even something like json.
I think s-expressions were just the absolute simplest way to represent them with the smallest parser required. It's really really not much.
tl;dr the killer feature of s-expressions is that they're trivial to parse and generate, which makes it easy to read/write some other "front end" instead (i-expressions, sweet expressions, wisp, etc.).
This is really amazing and it's too bad the author is getting so little from his Patreon for making it! But at the same time I understand why. People can't contribute to every open source project that increases their productivity, there's just too many of them. But what other options do innovative geniuses like Shaun have to keep making things like this? Make a kickstarter for a product? Nobody would bite for small but invaluable utilities like Parinfer. Commercialize it and keep it closed source as an IntelliJ plugin? Would be incredibly hard to market. Keep it open source but charge for support? It's not complicated enough, he'd have to add a gazillion configuration options and intentionally bloat it just to justify it. I would really hate to see things like Parinfer die out because most people wouldn't justify spending the time making it, or would make it but wouldn't open source it.
>This is really amazing and it's too bad the author is getting so little from his Patreon for making it! But at the same time I understand why. People can't contribute to every open source project that increases their productivity, there's just too many of them
That's not a valid reason. They can also pick and contribute to their 3-4 more important, or to their 3-4 favorites, or to those that look more in need (eg. not much need do donate to Mozilla for example).
But how many Lisp programmers are out there anyway, and use this? The intersection should be quite small, compared with most famous FOSS projects for languages like Java or Python (and even those projects don't get much).
"People can't contribute to every open source project that increases their productivity, there's just too many of them."
I'm sure there are people on HN who can do just that, and much, much more. Some of them may even use parinfer. But whether they can and whether they want to contribute are two different things.
For me the more interesting question is how the tiny minority of people who have gotten their hands on the overwhelming majority of money, power, and resources in this country and around the world can be persuaded to use what they have in socially constructive ways rather than for the benefit of the tiny, exclusive minority they are part of.
- lists all the projects you depend on that support donations
- shows you how much (or rather little!) they're making now
- shows you the (Patreon) rewards you can get for supporting them
- allows you to support them right there and then!
I honestly think this is a legitimate idea and kind of want to pitch it to Y Combinator right now. It would be like something that ties Patreon and Github together.
---
Okay I just posted this idea to Hacker News, find it in my submissions if you're interested :)
Chicken v Egg! Haha, did not think of that :D Actually I think a Kickstarter would be more appropriate for this, but even then it seems unlikely to get any funding. I think going straight to a VC would be my best bet.
I don't think so? The product is too simple to get destroyed from money pressure and bloat. All it needs to do to make it worth their while is skim a little off the top, maybe 1%.
Very similar, but a product like this has to remove absolutely every point of friction it can, and make it as close to 1-click as possible. This is a command line utility that gives you links, which is a great start. We can make it even simpler!
> For me the more interesting question is how the tiny minority of people who have gotten their hands on the overwhelming majority of money, power, and resources in this country and around the world can be persuaded to use what they have in socially constructive ways rather than for the benefit of the tiny, exclusive minority they are part of.
That's what taxes are for. "Give up some of your money to be used for the collective benefit, or we will lock you in a box and then take your money away anyway" is pretty persuasive.
>For me the more interesting question is how the tiny minority of people who have gotten their hands on the overwhelming majority of money, power, and resources in this country and around the world can be persuaded to use what they have in socially constructive ways rather than for the benefit of the tiny, exclusive minority they are part of.
Income, corporate, payroll, VAT, property and capital gains taxes are great for this.
It's not making much money because it's not that conducive to writing real lisp code with. Nobody thinks about sexprs the way parinfer does unless that's their first encounter with them. If you went and taught parinfer users how to use paredit, they'd wonder why paredit is so much easier and feels more w useful to them.
Additionally, it's a niche market in the first place. People aren't exactly clambering for more lisp dev tools except in recessed corners of the academic universe and places where clojure is used. Aside from those users, there's little demand for a tool like this, and that market is like I said, further eroded by significantly higher quality tooling that's already available.
Not every piece of code someone writes is worthy of being paid for. Not every piece of code written deserves to have been written in the first place, unfortunately.
Most JS devs know sindresorhus by reputation. You can't help but knowing him through his work. He even put up a separate twitter solely because people kept wanting a way to see whenever he made a new github repo.
Patreon makes him $3300/mo. So that appears to be the ceiling for all of us.
I think the open source model that has been shown to work is to make a great code product for free and then add a few more awesome features for a “pro” version with support attached. That’s how Sidekiq works and he’s making good money:
https://www.indiehackers.com/interview/how-charging-money-fo...
> Patreon makes him $3300/mo. So that appears to be the ceiling for all of us.
Hopefully it becomes an average, not a ceiling. We're not all going to make the next Vue, but that guy is getting $16k/mo https://www.patreon.com/evanyou .. that's what I consider the ceiling.
I'm using the atom parinfer plugin for my clojurescript projects and it makes development so much easier! The "it feels like writing python" analogy really fits.
Now, if only there was a jump to function definition by ctrl+lmb for clj/atom it would be the perfect environment!
Also coming from Python and now using Parinfer for Clojure/ClojureScript, but I use the implementation in Cursive (intellij Clojure plugin). Love it. The only problem is working on a shared code base where people don't bother to indent code properly.
Thanks, that's a great suggestion, although I think this I'm probably not going to get it through. This is a team of probably around 50 Clojure developers using different editors and AFAIK I'm the only one using parinfer.
I've sadly never gotten around to using Lisp, but even still, this kind of thing excites me. I think there's a lot of untapped promise in supporting structure within free-flowing text editing, like this does.
I found this to be a great intro to the ideas of lisp. He covers macros, the Y Combinator, the meta-circular evaluator, and even a 62-line implementation of lisp in lisp. I’ve read about these ideas separately but had never seen them all together in one presentation, with examples. https://youtu.be/FRaNUsiD_BA
The lego analogy is something I've thought about before. Inspired by Firefox's Tilt 3D[1], I've wondered if it makes sense to code in VR with indentation, or even level of the AST projected onto the z-axis. Wobbling the perspective around would probably be annoying and unnecessary, but hopefully a fixed angle, or one varying slightly with zoom could work.
>More than anything else, I think it is the ability of Lisp programs to manipulate Lisp expressions that sets Lisp apart. And so no one who has not written a lot of macros is really in a position to compare Lisp to other languages. When I hear people complain about Lisp's parentheses, it sounds to my ears like someone saying: "I tried one of those bananas, which you say are so delicious. The white part was ok, but the yellow part was very tough and tasted awful." – Paul Graham
Lisp programmers don't deal with "parens hell". That's one of the misconceptions that come from outsider looking in. Parens are for the editor. Lisp indeed uses indentation and not parens as you said. It's very python-like that way."You have reached Lisp enlightenment when parens disappear"
I think it would be real helpful to have editor for beginners that shows parents as transparent and low contrast, separately from the rest of the code. It would help people to understand they should not be trying to match them visually.
Educated guess: because you have zero experience developing an application or utility program in a language in the Lisp family. Because then if you still harbored a caricature, it would probably be more sophisticated.
Because it’s just that, a caricature. Having spent enough time using sexps (primarily in Clojure) I’ve found that with a decent editor it is about identation and line breaks, which is generally true of most languages.
Parenthesis are one of the most elegant things about Lisp, and I would hate to have to live without them.
When I finally got around to learning Python, I found that its meaningful whitespace usually wasn't as bad as I'd feared, but it's still a pain in the ass in certain circumstances.
For instance, there are plenty of places online (like some blogs) where python code is published that screws up its whitespace so that copying and pasting from there totally screws up the program and I'm forced to fix it by hand. This is an unnecessary pain that explicit delimiters like Lisp's parenthesis solve.
Python's meaningful whitespace is also not enough for editing software to easily infer and manipulate the program's structure without actually knowing python. That's a burden that editors that are editing Lisp do not have, as they're always just going to be manipulating sexps, without needing a deep understanding of the underlying language. So that usually makes sexp-aware editors (and plugins like pariner) much more powerful and flexible when editing Lisp than your typical python editor.
Python also has way too many other types of delimiters. Though modern Lisps also don't have just parenthesis, most of their delimiters are parenthesis and that makes thinking about which delimiter is appropriate in which context a lot simpler. You have a lot less rules to remember, and that's a blessing if you're a polylingual programmer who's bouncing around from one language to another -- you mostly don't have to remember which delimiter has to go where, as it's mostly just a parenthesis with Lisp.
Instead of trying to improve Lisp by taking away parenthesis and adding meaningful whitespace, I'd rather improve Python by taking away its meaningful whitespace and using parenthesis.
> I love indented languages. Python, make, coffeescript etc indentation communicates hierarchy. It’s a very simple idea.
> All I’m saying is Lisp is amazing but parens aren’t for everyone.
It's trivial to write Lisp using indentation and no parentheses if you want to. The same goes for any language which uses s-expressions. Just pick an alternative syntax, e.g.
These can all be converted back and forth to s-expressions, such that the programmer doesn't have to care about s-expressions (found some code online? send it through a converter before reading it), and the language implementation doesn't have to care about the syntax that the programmer is using (set up the relevant converter as a preprocessor).
Two things to note:
- The reason we can do this is because s-expressions are such a useful language format. Using an alternative syntax for Python, make or coffeescript would be a time consuming, maintainence heavy burden. Anything using s-expressions gets all this "for free".
- The fraction of Lisp programmers who do this is vanishingly small. This is empirical evidence that parentheses are not a problem worth caring about.
As a lisp programmer at home, python programmer at work:
>I strongly believe python got so popular because of its very simple to read, almost pseudo-code like syntax that anyone could learn.
Agreed. And python programmers are asymptomatically approaching Lisp's features. I had previously thought that only Lisp dialects could do symbolic differentiation and integration on code written in the same language, but I was wrong.
http://www.sympy.org/en/index.html
> List had great ideas, everything is a list, but dealing with parens hell makes me think it could be a lot more elegant.
Lisp is already the most elegant. In John McCarthy's 1958 paper, he wrote a Lisp interpreter in Lisp, and the interpreter was only a page or two. That is elegance.
(This is not to belittle the point you are making, I just can't resist being That Guy. Lisp manages to be both a usable language and pretty easy to interpret, whereas Brainfuck is... something else.)
