JavaScript query language to perform complex object searches | Hacker News Reader

JavaScript query language to perform complex object searches

github.com

52 pointsjviotti21 comments

Open articleView on HN

Looks similar to MongoDB query syntax [1]. Is it inspired by that? What are similarities/differences?

[1] http://docs.mongodb.org/master/tutorial/query-documents/

Yes it looks very similar, having built a query builder for it: http://youtu.be/fZ3_NGLKE2U

Yeah, it was inspired by MongoDB, but then deviates in some ways:

- Conjunction and disjunction operations are defined by arrays in MongoDB, but as Objects in Queryl.

For example:

{ $or: [ { foo: { $gt: 100 } }, { bar: { $gt: 200 } } ] }

vs:

{ $or: { $gt: { foo: 100, bar: 200 } } }

- As you can notice in the above example, in MongoDB, you declare the property and the specific query inside, while in Queryl is the opposite: you declare the operation, and then the property.

- Of course, Queryl is still in it's infancy, and doesn't support as many operations as MongoDB does (but will be added eventually).

There are probably more differences that I'm not spotting due to lack of in depth knowledge in MongoDB queries.

I have been enjoying https://linqjs.codeplex.com/

I've been working on library that creates a queryable structure for trees like this. I prefer writing normal JS than to having some query language for it.

  var obj = minim.toElement({
    a: 1,
    b: [
      {
        c: 1,
        d: 2,
        e: [ { f: 5 } ],
        f: 6
      }
    ]
  });

  // Results are [5, 6]
  var results = obj.find(function(value, key) {
    return key.equals('f');
  });

See https://github.com/refractproject/minim

An alternative approach is to use XPath-inspired DSLs that work on JS or JSON object structures. These tend to work on strings, rather than declarative query objects. Examples:

JSPath: https://github.com/dfilatov/jspath

JSONPath: https://github.com/s3u/JSONPath

(Since you don't have type safety in JS, I don't think you lose much by using strings, unless you want to wrap it in TypeScript.)

Or you could actually use XPath :)

https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/...

This could use a touch more intelligence to cut down on a lot of the noise.

You could eliminate `$equal` and `$match` by looking for regex types and strings.

You could also take hints from Django and add operations to the keys to eliminate instantiating a new object for `$not`, `$gt` and `$lt`.

So, instead of:

    queryl.match({
      $or: {
        $equal: {
          foo: 'bar'
        },
        $and: {
          $not: {
            $match: {
              foo: /^baz/
            }
          },
          $gt: {
            bar: 3
          }
        }
      }, ...

...it would look like:

    queryl.match({
      $or: {
        foo: 'bar', 
        $and: {
          foo__not: /^baz/, 
          bar__gt: 3
        }
      }, ...

Seems like this syntax is popular; is there something about this that doesn't work when serializing, or other uses that need the pure-object approach?

Hey there! Your second example was the original idea when building Queryl, however that required me to always inspect the properties of an object, and based on the key, check if it is a Queryl operation (e.g: starts with $ and it's recognised), or a simple property to match for equality.

Also, it had the edge case in which a user's object properties could collide with the operations defined by Queryl, which (I believe, didn't write tests for it) doesn't happen with the current approach.

Can anyone explain why you might need to store queries as JSON data with horrifying syntax, instead of just writing plain functions that search/filter/whatever?

Or if you really have to, using a sensible DSL to do the job instead?

Hey there,

Objects (or DSL) are easier to encode/decode than functions.

Also notice that Queryl is a very low level way of defining queries, and can easily become horrifying and unmaintainable with very complex queries.

The idea was to provide the low level, verbose way of defining this, and provide a user friendly frontend (as a DSL, probably) that compiles to it.

Also similar to https://github.com/raine/ramda-cli which was posted recently.

Along similar lines, I just made this library a couple weeks ago to describe data validations programmatically (i.e., one JSON to validate another JSON document).

https://github.com/MalcolmDwyer/json-predicate

Looks similar to sequelize http://docs.sequelizejs.com/en/latest/

Looks similar to sequelize query API, direct link: http://docs.sequelizejs.com/en/latest/docs/querying/

This kind of things looks better in homoiconic languages.

I'm wondering how that approach compares to rho/rewriting calculus.

Looks cool, I haven't heard of it. However I don't think I could use their querying language separately from the ORM.

Our use case for Queryl is to filter array of objects based on different criteria, but I guess an extra benefit is that the query language is completely decoupled from its usage, so anyone can use it in their own projects for different needs.

looks very similar to siftjs: https://github.com/crcn/sift.js/tree/master

It always feels like we keep reinventing Lisp. You could do something like:

  ['contains?', ['list', 1, 2, 3], 1]

Instead of:

  {
    $contain: {
      foo: 1
    }
  }, {
    foo: [ 1, 2, 3 ]
  }

Haha definitely! You could write a small module that compiles the lisp-like syntax that you mention above to a Queryl object.

how's this better than just using lodash?

The project heavily relies on lodash for it's implementation actually, so you can of course do it straight with lodash, however the benefit is that you don't have to write any code to produce complex searches (with multiple nested levels of $or and $and), which would require a lot of conditionals if done straight with JavaScript.

Another benefit (which was the reason behind the implementation), is that we can stringify the search object to easily persist and share it, which would not be that easy with plain code.

Reply on news.ycombinator.com