To me, pipe feels far more intuitive than callbacks as a way to write complex queries. Using it right away might make the first few examples much easier to read.

I suppose to each their own. I like how the query matches the json structure well.

Hi, awesome work! Love the idea of using S-expression for the query syntax! Funny fact: we share the same project name - even though the core language and the query syntax are different - see https://github.com/yamafaktory/jql.

Hey, looks really nice! I see we had similar goals of simplicity in mind, though with very different approaches.

Sorry for the name duplication! I thought I'm sharing the name only with the jira API jql.

EDIT: I'd love to see some benchmarks. I expect your jql to greatly outperform mine and jq, considering it's written in Rust!

No worries regarding the naming, it's what makes open-source friendly! You can see some benches here e.g. https://travis-ci.org/yamafaktory/jql/jobs/618374730#fold-st... but those actually more like regression tests that are triggered for pull requests to compare them against master (and I should use Github actions probably here now instead of Travis). Thanks for your reply!

Looks good but there seems to be a missing explanation gap between switching from `(elem "countries")` and dropping the `elem`s to just `("countries")`. Why is `elem` able to be dropped? Is it the default function or something? I didn't follow that big jump in the readme.

This feature can probably be thought of similarly to Clojure's map access via keywords: if you write `(:something map-var)`, you'll get the value of the field ‘:something’ in the ‘map-var’ variable. I.e. keywords can be ‘executed’, and such calls get translated to map access.

Indeed, it just extends to values, and works recursively if it's an array or map.

It's explained right below, I think I messed up the order there.

Is this understandable? "You can see that elem is the most used function, and in fact it's what you'll usually be using when munging data, so there's a shortcut. If you put a value in function name position, it implicitly converts it to an elem."

EDIT: Fixed the order.

Nicely done!

I scratched a similar itch by creating Jowl[1], which uses JavaScript one-liners with the Lodash library to transform JSON.

Like you, I wanted syntax that was more familiar to me than jq's, and like you, I wanted to use a language that was built for data structure transformation.

However, I offloaded all of the actual hard work to an existing JavaScript runtime, and to an existing library for data structure transformation, meaning I could punt on the parser and language design; the hard parts that you've taken on. I have this bookmarked to read through in more detail later this week.

[1]: https://github.com/daxelrod/jowl

Sounds great! I'll make sure to check out out!

My approach was caused by the fact that I wanted to have a minimal language which only contains what really is necessary and is as regular as possible, to be very simple as a result.

Looks great!

As an old Lisper, I did get a wry chuckle out of the fact that you managed to avoid Lisp for a tool that uses Lispy syntax. :-)

Thanks!

Yup, I felt the irony myself creating it, as I actually love clojure!

But I'm most comfortable with Go, and it results with a single binary for any of the major platforms with very quick startup time (as illustrated by the small benchmark), so those were huge advantages.

I love the idea of a Lispy version of jq, but as a Lisper, the examples feel bewildering to me. The fact that (elem) takes an optional second argument, whose default value is 'identity, seems familiar, but then I try to understand this example, and it just doesn't make sense to me:

    (elem "countries" (elem (keys) (elem "name")))

I think part of the problem is, IIUC, this is constructing a pipeline, like (->) in Clojure, but is using function call syntax rather than pipeline syntax, and also mixing in a function call with (keys). I feel like it's a very non-Lispy language masquerading as Lisp by wearing parens.

Instead, if you wrote it as real Lisp function calls, and provided a real pipeline macro, it would be much more Lispy and, I think, much easier to understand. For example, instead of:

    (elem "countries" (elem (keys) (elem "name")))

Either of these alternative syntaxes:

    (-> "countries" (key "name"))
    (key "name" ("countries"))

Would return the same result:

    ["Poland",
     "United States",
     "Germany"]

While being more concise and more Lispy.

I also wouldn't mind using an anaphoric 'mapcar, something like:

    (-> "countries" (map-> (key "name"))

Otherwise it may be unclear which functions implicitly map across entries and which don't.

My two cents. Thanks for sharing your work.

Thanks for the feedback!

Have you checked out the type cheatsheet? (keys) is a function call!

The function you pass as last to elem is kind of a continuation. It's a function which transforms the output before returning.

All you're doing writing a jql query is composing one big function.

With values in function call position, it's just that (keys) gets evaluated to a value (the keys of the map) and that is used to index the json. That's why keys wouldn't work, you need the parentheses to make it a function call.

This may sound complex at first but I find it gets intuitive quickly.

EDIT: Check out this comment thread on reddit as I think the commenter may have had similar problems: https://www.reddit.com/r/golang/comments/ehnsz5/jql_json_que...

> Have you checked out the type cheatsheet?

I'm afraid that that does not help me understand how (elem) works at all. From my perspective, I don't need a "type cheatsheet," I need elem's docstring that explains what arguments it expects and what it returns. I'm not even thinking about value types yet, and I don't know what grammar that cheatsheet is written in, nor why I would need to read one to be able to use what seems like the most basic function jql has, the elem function.

To write clear, useful documentation, you need to put yourself in the shoes of a user who has never seen your project before, who has not walked the miles you have to arrive at the solutions you have, and take them quickly to the same destination.

The readme, in general, is another issue. It's...not ideal.

The first part tries to sound cute with stuff like, "Hey there!" and, "remember? That's explicitly not why we're here. But it aids understanding of the more complex examples, so stay with me just a little bit longer!".

But I'm not a little kid who's bored in math class, so writing like that turns me off quickly. Having to read through long prose like, "Ok, let's check it out now, but first things first, you have to install it:" instead of simply a heading titled "Install:" feels like I'm wasting my time, and I quickly skip ahead or move on to the next HN article.

What I'm really looking for is a short intro and a table of contents with sections like: Intro, Examples, Tutorial, Reference, FAQ.

You might think of it like this: the project is your baby, but just like in real life, no one thinks your baby's as cute as you do, and, no, we don't want to see the baby pictures. ;)

I actually don’t think like that. I even have another fairly successful open source project whose readme is probably more to your liking, and is nothing like this one: https://github.com/cube2222/octosql

Everybody has different taste. I like README’s like that as they add an entertainment value. Overall I’ve got two kinds of feedback about the README: 1. I really like the project and really liked the readme, great fun to read! 2. The readme was very confusing on top of the already confusing query language.

And based on that I'm planning to keep it unchanged for now. I added the type cheatsheet to make case 2 at least a little bit better.

As for the quick examples, you can just scroll through the code blocks (as each one contains input with its corresponding output).

Since you're discarding feedback kind #2, and all of the potential users it represents, in favor of entertainment value, I'd say that you do think like that. ;) Comedians write lots of jokes, but when they stand up and tell them in front of an audience, they find out which ones work and which ones don't. How they respond to that feedback determines their success. Hey, it's your project.

I'm definitely not discarding it! I'm open to writing additional clarifying paragraphs. (that's why I wrote the type cheat sheet). I'm also not discarding feedback kind #1 :)

I'm thinking of adding another one which shows how standard map/filter notation maps to jql queries.

Anyways, thanks for the feedback again, I do appreciate it!

Regarding CPS: I think that expecting users to write queries in CPS is going to be very limiting. It's missing a big opportunity to use simple funcall and/or pipeline syntax, with which many more users are familiar. Even among Lispers, CPS isn't that popular. But Lisp, in general, could make jq-like syntax much more regular and usable.

I find it gets intuitive without being familiar with CPS (I wasn't before).

In practice, thanks to this, the resulting queries end up matching the input JSON structure very well.

I was seeing it as roughly:

  (elem x f)

is sugar for

  (filter-on f (elem x))

or so? (filter-on imaginary thing but hopefully obvious enough as pseudocode)

Almost! It’s more like (map f (elem x JSON)), where map gets changed to apply, if (elem x JSON) results in a single value (non-collection).