Ebay launches query language | Hacker News Reader

Ebay launches query language

ql.io

81 pointsjurre29 comments

Interesting, another use of a SQL-like language as the query language for a data store that's definitely not an SQL database. (YQL is another example.) I really don't understand why ORMs always seem to want to create their an object-based query language--SQL is just fine for querying.

It's difficult to provide tooling for SQL in a way that provides a good experience for the user. For example, providing auto-completion in a SELECT statement is hard because attributes are specified before the tables which contain them are declared, leading to a-more-annoying-than-it-should-be editing experience.

Interesting. Isn't this the metalinguistic paradigm in action? Ie instead of creating a library/API you create a whole new Domain-Specific Language.

Thoughts on this? Is there room for innovation in APIs via DSLs?

It would be cool if there was some kind of meta-DSL which allowed you to create your own DSL. That way, more developers could go that route without having to learn all the intricacies of compiler theory and stuff like that.

I am imagining DSLs being useful both on the server side and the client side. For the former, imagine sending off a single query instead of doing a bunch of different API calls. For the latter, well, JQuery is an existing example of a popular client-side DSL (technically it's not, but it feels that way).

It's called Lisp. I hate to be Yet Another Hacker News Lisp Punditparrot, but it's true.

For example, take a look at how easy it is to implement BrainF inside Racket, a lisp focused on language experimentation. http://hashcollision.org/brainfudge/ I bet you could walk through these steps in maybe half an hour with little to no prior experience.

eh easy? that's one of the longest pages on the Internet! Here's a BrainF interpreter in 1 line of Python: http://www.cs.princeton.edu/~ynaamad/misc/bf.htm

Granted it expands to more than one line, but I'm not seeing that as a good example of lisp's superiority.

It's absolutely not the same thing : the Racket example is not a Brainfuck interpreter, it's an actual DSL inside Racket, which benefits from it's whole infrastructure (JIT compilation etc.). It's not the Racket language running a BF interpreter. It's the BF language running.

I hold that it's far easier to read and step through Danny Yoo's BrainF interpreter than to try and piece together your one-line pythonic monstrosity.

Of course, if you're arguing that your Python version is somehow better than the longer idiomatic Racket implementation, I think we both agree that leveling the playing field would create a more fair comparison. Heavens, wouldn't it make more sense to compare a one-line BF interpreter written in Racket to your one-line BF interpreter written in Python? ;)

Two can play this game:

  #lang racket (require racket/match) (let* ([data "++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>."] [move-left (λ (left cur right) (let ([right (if (null? right) '(0) right)]) (list (cons cur left) (car right) (cdr right))))] [move-right (λ (left cur right) (let ([left (if (null? left) '(0) left)]) (list (cdr left) (car left) (cons cur right))))] [inc (λ (left cur right) (list left (add1 cur) right))] [dec (λ (left cur right) (list left (sub1 cur) right))] [consume-one-char (λ (char state) (match char [#\+ (apply inc state)] [#\- (apply dec state)] [#\< (apply move-left state)] [#\> (apply move-right state)] [#\. (begin (display (integer->char (cadr state))) state)] [#\, (list (car state) (char->integer (read-char)) (caddr state))] [else state]))]) (let loop ([counter 0] [prev-counters '()] [state (list '() 0 '())]) (if (= counter (string-length data)) state (let ([char (string-ref data counter)]) (match char [#\[ (loop (add1 counter) (cons counter prev-counters) state)] [#\] (if (zero? (cadr state)) (loop (add1 counter) (cdr prev-counters) state) (loop (car prev-counters) (cdr prev-counters) state))] [else (loop (add1 counter) prev-counters (consume-one-char char state))])))))

Both this example and your one-line python example are similarly incomprehensible. While we're on the topic of meaningless benchmarks, please allow me to point out that my racket version is 163 bytes shorter than your python version, without trying to abbreviate variable names and without smashing whitespace. That's even including the "Hello World" BF program near the beginning of the source.

Ok, yes, you have a point -- the article I cited earlier is long and focuses on Racket's awkward tooling. However, my point is that we're comparing apples to oranges here. It would be better to compare Danny's article to a Python article that implements a BF interpreter, covers some optimizations, and describes how to package it as an Egg and upload it to PyPi. None of these topics are trivial, especially to a beginner, hence Danny's thurough coverage.

> Granted it expands to more than one line

Does it? It's a single semicolon-free Python expression. Looks like one line to me.

Besides, that is a terrible example of a "good" Brainfuck interpreter. The Scheme example is easily 100x easier to understand (/debug/troubleshoot/extend/etc).

How would I go about setting up Racket as a backend? Ie something for my JS or whatever to send requests to in my custom DSL.

Nice. Is it possible to cross-compile it to other languages?

I... don't think so. Not in the way you're thinking of, anyway. In this BrainF example, your language "compiles" (translates, really) to Racket expressions; then, the Racket VM "compiles" it to Racket bytecode just before running it.

For something as simple as BrainF, it's not hard to write your own translator. Just replace every [ with while (* ptr) {, replace + with (* ptr)++; replace < with ptr--; and so on. Then it's essentially a BrainF-to-C translator.

It's not so simple for racket in the general case. Danny Yoo is working on "Whalesong", a javascript interpreter that interprets racket bytecode so you can run racket in your web browser. http://hashcollision.org/whalesong/ It's still not a Racket->Javascript translator though.

I'm not aware of similar projects for other languages.

(Edit: formatting ate my asterisks)

Anyone who implements this might want to bear in mind that you will have created a very chatty backend based on JSON calls.

It should be fairly obvious that you're quite likely to now have pain points in serialising and deserialising that much JSON.

It seems to me that you now have 2 choices:

1) You forget this approach and instead perform REST composition and aggregation by adding a datastore in the composition/aggregation service to reduce the number of calls to the back end.

2) You ditch the internal REST services and change to using protocol buffers or something else to remove the JSON bottle neck internally.

Either of which will make ql.io redundent.

ql.io appears to be trying to solve REST as the language of enterprise services, without using a service bus or BPEL to aid with composition or aggregation.

I agree there is a problem space here, and whilst I have an immense amount of respect for Subbu (worked on ql.io, authored REST book) I'm unconvinced that this approach is the solution to the problem.

Ah, looks like this is the library Subbu Allamaraju refers to at http://www.subbu.org/blog/2011/11/apis-are-a-pain--the aim of ql.io is to alleviate the challenges of "aggregation, parallelization, and orchestration of forks and joins." (i.e. in practice, "consistent RESTful APIs don’t matter as much as we think," nice though they may be.)

I've been thinking lately that certain APIs would benefit from two distinct 'entry points':

1. a nice, object-oriented hierarchical RESTful interface for standard queries and getting started e.g. /api/user/12345/friends/

2. a sql-esque query language interface for arbitrary, complicated requests e.g. /api/?q=select id from users where name="Joe"

It looks like ql.io could be a good way to provide the latter.

Right. One of ql.io's goals is to enable (b) but on a tier between clients and servers.

Why, oh why did they put "select" before "from"? It makes auto complettion so much harder.

easier to remember that format, other sql formats are same order...

Exactly. We tried to stay close to the SQL syntax to a large extent although there is no relational storage underneath.

This looks a lot like OData... strange that they don't mention it when they compare themselves to other querying technologies on the 'about' page.

http://www.odata.org/

OData specifies a resource model, resource metadata and URI naming conventions for HTTP APIs. The query formats in OData are part of the producer-implemented APIs as queries happens at the origin. ql.io is a layer above such APIs, and generally operates across several such APIs.

Site's down for me

For me too now, theres some info on their github page: https://github.com/ql-io/ql.io

At first glance, it looks a bit like a host-your-own alternative to Yahoo!'s YQL, written in Node.js

Ah, thanks for the github link. It looks like an interesting project although I'm not sure what it brings over a more 'traditional' approach to API consumption

What I got from their website earlier is that it's meant to reduce the number of API requests by bundling multiple datarequests, giving you better performance.

Having the "fork me" button above the scrollbar (at least in Chrome on Windows) looks pretty ugly.

Thanks for the "hint". Checking.

Yahoo Pipe ?

Reply on news.ycombinator.com