JrGQL, a GraphQL alternative
jrgql.github.io
jrgql.github.io
Yes, APIs seldom elegantly encode into the set of HTTP verbs and responses that we associate with a "RESTful" design, I grant. And so maybe we can come up with better.
But the notion of JrGQL and GQL as query languages means that the servers handling these calls must be query resolvers. Unlike most restful interfaces which tend to devote a single uniform interface per endpoint with only minor modifications, a full query model of your domain means an explosive quantity of potential strategies piped through a single endpoint.
I've used Python, Ruby, Node (in ts and js) and Haskell to service GQL queries and in all cases it's not trivial.
The popular NodeJS bindings tend to cause huge overfetching because each field tends to have a unique resolver but there is no rule about combining them. The pooular Python bindings (graphene) let you merge this, but the programming model to handle the arguments and sub-arguments is very frustrating as in different places, different soirces of logic will government what gets fetched (sub objects use SqlAlchemy, but outer objects with ANY sorry of query logic need to be custom). Ruby's bindings are the same.
Haskell's popular solution let's you cobble a responder from a proof of concept, and it leads to optimal query scheduling. Still not the best: it's by no means complete and requires quite a lot of work to set up.
These GQL systems push a huge burden onto every api endpoint with the proposed trade-off: "Well now the client has an easier time." Even if that's true, now the backend needs to be much, much smarter than before to give a marginally better interface for clients.
I'm still very skeptical of this whole concept.
Regarding nodejs and overfetching, the canonical solution is Facebook's DataLoader library. I think it might even be in the docs somewhere.
Regarding jrGQL, I'm not sure the use case is the same. They look like different tools for different problems, but I'd be interested in hearing more about what problem it solves.
From the server point of view, using GraphQL may not sound great since any single tiny update on the schema also implies an update on the single endpoint, rather than just one of many. Until there is an implementation of dynamic modularization, maintaining the integrality of a service is a constant pain.
Fairly enough, I think the tradeoff is balanced. GraphQL solves problems on the client side and yet it creates problems on the server side.
Meanwhile, I am more concerned about the future of GraphQL as http2 has arrived. With http2, the biggest selling point of reducing latency by using one single endpoint does not sound anymore. Then for the flexibility of limiting the scope of return, you can always specify it on the conventional RESTful API. So there is not much practical benefit. (Oh yes, there are some more goodies like subscription, but let's face it that not many use it) So to make GraphQL more useful, I think it needs a bigger evolution.
GraphQL's most significant latency gain isn't achieved through the use of a single endpoint to minimize parallel requests. As you mentioned, HTTP2 makes this a non-factor in terms of latency. The single endpoint part is only an implementation detail, and most follow it simply by convention.
Rather, GraphQL's major breakthrough in latency reduction is achieved through the use of the graph query language to completely bypass the multiple client-server roundtrip problem for interdependent data (i.e. having to wait for a user request to return the organization id, before being able to send a request to find some property off the organization the user belongs to). HTTP2 isn't able to help at all for this use case.
Well I've yet to work with a set of server-side tools that actually deliver on this promise. I really don't like the tools I have.
I'm now attempting to synthesize and optimize SQL queries in python. Messy business. Way more defect-prone.
> DataLoader
I'm a haskell programmer and even I think that programming model is pretty brutal.
* It's fairly easy to tell "how the app is using the database" -- queries are not dynamically generated or hidden in code.
* The query performance is predictable and controllable in the same way as a stored procedure approach to database access -- but without actually requiring a DBA to load/unload procedure definitions.
One might argue that these benefits could be obtained without codifying a new query language...but when you start to think about nested objects you start to warm up to the idea.
You do, but any time you try to solve a problem, and your solution ends up being a new language, I think it's worth asking yourself, "Am I overcomplicating this? Can I do this in a simpler way?"
And I think you can. I've been toying with an approach I like to call http-etc, which is a simple idea that lets you flag the http (rest) api to say, "Give me this data, but also give me the graph associated with this data."
For instance, if I want an article, I would do:
/api/0.1/article/z098d0s8dga
But if I want an article, plus the comments associated with it, and information about the users that posted those comments, I can do:
/api/0.1/article/z098d0s8dga+
And this is handled completely with your server-side api design. You have your singular route to one function, and you have your "etc" route to another function. Synchronizing the shape of the data requested/returned between server and client is still handled manually, but it seems to me this pattern solves 90% of GraphQL use cases with 1% of the complexity of GraphQL. It isn't tied to any specific language or codebase on the server-side. It doesn't require a big client-side library be added to your project. It's just a simple pattern.
The point is exactly to push data fetching complexity from the Client to the API server. And that's generally a worthwhile tradeoff for these applications because you can deal with server complexity by throwing more money at your API server cluster, but you can't force your users to upgrade to more performant clients.
Of course, server-side overfetching is a problem that can drastically limit the scalability of your system by overloading the components you can't as easily scale by spinning up more machines (i.e. most databases). Naive implementations of GraphQL on the server can be ridiculously chatty and require multiple server-db roundtrips to resolve even the simplest of queries, which can arguably be an even nastier problem than dealing with multiple client-server roundtrips with a RESTful API.
This is why non-trivial GraphQL servers generally use some kind of resolver batching/caching layer (Facebook provides a library called DataLoader to facilitate this: https://github.com/facebook/dataloader) or a query planner at your root resolver (like Join Monster: https://github.com/stems/join-monster).
Neither of these approaches is trivial to implement, but then again, doing efficient data fetching on the client against a RESTful API is just as difficult, if not more so. Efficient data-fetching is just a very difficult problem, and involves essential complexity that needs to live somewhere. At the end of the day it's up to you if you want to deal with that complexity in your clients (using a traditional RESTful API) or on your API servers (using a query mechanism like GraphQL).
For example, server side rendering. Though Hyperfiddle is a bit more sophisticated than that.
I'm now completely sold, for a JS powered app, using Preact and Apollo-client makes working with the backend incredibly easy, and realtime via subscriptions works very well.
I've used it to build a couple of large apps, and am seeing very good results, and much easier to build something in the frontend. The backend handles these queries very well via pattern matching, and I also know the queries ahead of time. In future I would want to lock these down to certain queries.
I'm very impressed by GQL, after having to deal with crappy APIs in the past. JSONAPI was almost perfect to me (working with Ember), but the fact there was no bulk submit was very shity to me.
Gql buys me absolutely nothing at all having microservices connect.
I do! I find it odd that anyone would do anything else...
In microservices-driven-architrctures you often have to appeal to multiple services to service a request.
A single GQL query resolver must be able to recognize which field you want and ideally construct the minimum query that meets the requirements. This is much harder than simply recognizing the correct parameters out of the URL, body and queryParams and execute it.
While the stakes are low for overfetching in this case (odds are the DB doesn't notice and maybe it's not a problem for your network link), as a general problem (in particular when fields have subfields) this gets frustrating very quickly.
My concern is not that this is bad for clients. It makes a ton of sense for clients. It's just much harder for servers to schedule optimal queries. Suddenly you're not only managing unreliable resources and minimizing latencies you don't own, you're also a compiler or interpreter for a language with more than one valid query.
For some types of backing stores it may not matter (ECKRV's such as Dynamo, for example). For others it may matter a lot (Postgres). And I think a lot of server owners ship backends that maybe do 2x more querying than they have to, given the "best practices" and examples I see online for the NodeJs bindings.
I've heard the Erlang bindings are very good. I haven't tried them yet, as Erlang isn't very high on my list of tools to use these days (not because it's bad). Maybe they have a silver bullet.
The pain I've felt is with API versioning. Someone set a type for a parameter wrong, it was a union between int and string but the original implementation was only typed for int. Since it was sent out to production that becomes too risky to change since the call will fail if the param type is wrong, even though the underlying JS would have handled ints and strings just fine.
Overall it is a pretty neutral technology for me. The benefits and determents seem to balance just enough that I'm not going to pick that battle to fight against it. So while I agree it doesn't add as much value as the hype might suggest, it also doesn't detract enough to worry about.
With this approach you can end up with a somewhat noisy schema, but for public APIs it's usually worthwhile to just keep the old versions forever to avoid breakage. For private APIs, with good tracking, you can find out when the original field stops getting requested, and clean up your schema then with relatively little risk.
GQL does seem to encourage, if not assume, a type-theoretical graph datastore as the backend. (Who couldn't whip such a thing together, if it were lacking?)
But it does not require one. This point may be lost in the tutorial and examples shown on graphql.org. There is no reason one could not map a basic GQL service onto an existing set of REST endpoints, with each one a type, and the return of their fields controlled by the request.
I know you meant this to be rhetorical but unless this added some fundamental features (e.g., abstracting a distributed transaction negotiation) besides a new awkward interface I think I'd lose my mind writing GQL proxies over all my existing services.
Why just add a layer of waste and latency, along with a whole new intermediate serialization? Seems like make work. Just to adopt a technology for no real reason.
On the other hand, if your page loads information from ten different endpoints across three different services, and some calls depend on the results of previous calls, then you have four options: take a massive performance hit, go back to rendering everything on the server, create an endpoint for each page, or implement some way of querying embedded resources.
GraphQL acting as an intermediary for multiple REST apis is one way of doing the latter.
Also, http2 push is VERY interesting for application servers.
But the fact that I've only seen one implementation correctly implement the query semantics worries me deeply. At first I assumed I was using bad libraries! But soon I realized that lots of devs were simply shipping significantly less efficient servers to production.
a full query model of your domain means an explosive quantity of potential strategies piped through a single endpoint.
Sure, but if we're doing that by dragging the API implementer over every possible permutation of queries, optimizing whatever you could possibly ask for, I'm not convinced this is LESS work than me shipping BTO API requests.
> I'm sure certain types of projects will benefit more from the pros than cons of GraphQL.
My sneaking suspicion is that it's the kind of projects where their data model is small and isolated and doesn't offer much sharing between queries. Which is great when you can get it.
I don't have any experience with NodeJS+GQL but used Golang+GQL a lot. Typically a resolver of a field simply returns the property of a structure that is returned by the parent field, that's it. The only thing is the parent field should unboxed from the interface{} type.
We've been using REST and slowly adding hacks to stop sending so much data. We have 2x API calls that used to be 1mb each and now they're something like 5kb with GQL.
After implementing GQL, I found this very old slide show on REST and it happened to solve my complexity problems. https://www.slideshare.net/mobile/landlessness/teach-a-dog-t...
GQL and this alternative have their place and they solve problems for very large teams of (like the teams at Facebook), but for smaller teams and smaller apps, I'm not sure it's the right solution to REST's "join" problem.
First off, what are the advantages of regex queries here? One of the major advantages of GraphQL alongside libraries like Relay or Apollo is that there are strict, easy to understand data requirements that are tightly coupled to view logic. I know my server API, why would I want something like this? Especially in considering the regex keys incur a non-trivial performance penalty.
This page comes across misleading or disingenuous at worst:
1. "JrGQL" is listed as a non "new language" even though GraphQL's spec was published before it came out, and has been in use at Facebook since 2012. I'd imagine this is because jrGQL uses JSON. It still requires parsing on top of JSON. I don't know why the author thinks this counts as a non-new language.
2. All of the jrGQL "features" listed in green and almost all of the competitors in red or yellow? The page does nothing to claim why strict typing is a bad thing.
3. jrGQL is touted as more readable without any explanation as to why. I personally find it way more unreadable, as graphQL comes across largely intuitive to query even without knowledge of it (building performant gql servers is another story). For example, can author really claim that something like:
```
{ "// JSON RegExp Graph Query Language": "", "name": "jrGQL", "?[filter]": "", "search?": ".values$" }
// is more readable than: query GetAllTheThingsQuery { people(name: "GQL", search: "Smith", first: 5) { name } } ```
I see now that regex may be an overkill, the main intent was to support keys that are not predefined. It is easy to imagine a data set where a lot of keys can exist with the same prefix: { "text-decoration": "underline", "text-decoration-color": "red", "text-align": "center" } For this situation it may be useful to query all the keys with prefix "text-", without exactly knowing what are the possible keys. Since document databases and JavaScript are popular, we should open up the strict typing. Clients may have the right to store arbitrary metadata on their objects. In GraphQL you have this option, just add a metadata key as string and you can dump anything there, but this way you lose the option to filter on this. That exact missing feature inspired jrGQL.
1. Yes, it is not a new language, since it is always a valid JSON. This is useful since every validator, syntax highlighter already knows it. GraphQL created a new language just because they not considered to reuse an existing one. You can say that jrGQL still needs parsing on top of JSON, but every query should be validated on the server side and that needs parsing too. Also we can call regex evaluation as parsing, but this is just a more by a substring() call than simple equality check to filter the result set.
2. Coloring is definitely a mistake, I'll fix it. I did not want to say that others are bad in anything. Man, those are production ready quality and this thing is just a concept with only a reference implementation. I just wanted to show where jrGQL is different from others.
3. This is about tastes, I should add that for me it is more readable. I don't like GetAllTheThingsQuery stuff, since it is defined by an other party, if you need that it is an other endpoint and you get what you were meant to get, but no control. With jrGQL you always get what you requested, so it is not WYSIWYG, but WYRIWYG.
I just wanted to point out where jrGQL is different from others.
Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.
Although you and almost everybody else was pointing out regex as a possible problem or at least as an unnecessary complexity. I plan to handle this: https://github.com/jrgql/jrgql.github.io/issues/2
Now just marry the mutations to the GET/POST/PUT/DELETE operations of REST and I'll be smitten. No parent +/- node needed, and there are already a ton of REST libs out there.
though I haven't seen a gQL alternative that does a better job, Falcor is close
On the tech side:
- RegExps open yourself up to RegExp-based DoS attacks.
- The nested/denormalized results anti-pattern seems to be copied from GraphQL. Unless this is intended to be a faithful reproduction, don't copy mistakes.