Hi, i can give some background how i ended up with go instead of using something more declarative. Maybe 1.5 years ago i start to prototype different approaches for what query language to use (sql, jsonpath, my own basic jq version and few more) and what language to implement decoders in (lisp, kaitai, tcl, "scripted" go, normal go and some more).
What i found was that for my use cases, detailed parsing of big media files, anything scripted was just too slow. I did look into translating kaitai etc into something compiled which would probably be fast, but next on my list was i wanted was to be able to select and decode subformats in quite complicated ways (like mp4 samples), flexible ways of demux and join blob to decode, calculate checksums, samples counts in various way. All felt clunky or hard to fit into a purely declarative description.
But i was also biased towards go as i had good experience using it and know that it would probably be fast enough (turn out smart memory usage is probably the main speed factor for fq when you keep track of lots of things). Also it would provide good tooling like IDE support, refactoring (gopls gofmt -r, rf) and it's a reasonably strongly typed language i think. Last but not least the quick build times really fits my way of working, usually use lots of watchexec etc.
For query language i didn't prototype much, i know i really wanted jq as i had already used it extensively and know it was very powerful and had a terse syntax when working with structured data. I had some ideas of maybe using the C-version of jq via bindings or somehow let fq be tool that you used like this 'fq file | jq ... | fq' but it just felt strange and not very user friendly. Then i found gojq and i just felt that i have to make it work somehow, even if it would require lots of hard work and change to it (see https://github.com/wader/gojq/commits/fq, the JQValue change it probably to most interesting and support or custom iterators/functions that has been merged). And it turned out much better than i would expected, large parts becuse gojq's code is very nice and author has been very helpful.
There is more things i would like that talk about but i think this is long enough for now :)
But all that said i think you could use kaitai or something similar together with fq:s decode API if you want. I also have some ideas and plans on supporting writing deocders in jq, hopefully will get some time for that next year.
But all that said i think you could use kaitai or something similar together with fq:s decode API if you want. I also have some ideas and plans on supporting writing deocders in jq, hopefully will get some time for that next year.