What I really want is a version of this type of tool that's truly a single binary, that implements JSONPath, or some spec'd alternative that I can invest in learning, knowing it's not just a fad.
What I really want is a version of this type of tool that's truly a single binary, that implements JSONPath, or some spec'd alternative that I can invest in learning, knowing it's not just a fad.
RumbleDB is free, with no restriction on commercial use, and open source. It is an academic project, the product of now 6 years of work by 20+ ETH Zurich BSc and MSc students in their projects and theses.
You will find a video tutorial given at the Declarative Amsterdam conference last year here: https://www.youtube.com/watch?v=3YkLXQVyN2o (link edited, it was wrong)
RumbleDB works with small JSON files but also with several terabytes of JSON. It also works with many other formats; for example, it allows validating data and then storing it in a more efficient binary format (like Parquet).
It works on a laptop by spreading the computations on all the cores, but also in the cloud (it was tested on a cluster of 64 Amazon EMR machines manipulating billions of objects).
It can read and write data to and from a laptop's local drive, but also to and from data lakes (S3, HDFS...).
Feature-wise, JSONiq can do everything SQL can do (projection, selection, grouping, sorting, joins, etc), while also supporting heterogeneous and nested datasets seamlessly (normalization, denormalization, navigation, etc). 95% of it is directly based on a W3C standard including many of its builtin functions.
RumbleDB also has a Machine Learning library.
It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like.
I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99
I'm not saying I don't need to pull up the docs once or twice for regex or leafpaths (or was it leaf_paths, lol), but overall I can't think of something that needs to be pulled into the standard install more. It's near perfect.
[j/k. i like it too, but i'm sure you're still special. :D]