Living APIs and the Case for GraphQL
brandur.org
brandur.org
* GraphQL definitely seems to be more popular in the context of the client. A lot of the conversation is around client-side libraries like Apollo and Relay and how to consume APIs.
* Because of this, the actual GraphQL servers get less attention and there's less resources on building them.
* GraphQL on the client is a huge win in my opinion and I've never really heard anyone who's used it disagree with that.
* BUT, GraphQL on the server is absolutely much harder to write than a REST API. I've found two main reasons for this though:
1. GraphQL is more powerful and you don't get all that for free.
2. You end up paying more attention to designing a good GraphQL schema/API because of the type system.
JSON API and companies like Stripe with their `expand` feature offer equivalents to GraphQL but it's non-standard and way harder to do since you're inventing it each time. There's also no standard universal clients for these.
Greenspun's rule applies here in my opinion (slightly exaggerated):
> Any sufficiently complicated RESTful API contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of GraphQL.
I don't know where this will all end up. I do know that GraphQL isn't perfect and it's still early. Public APIs are tough right now because developers are so used to RESTful APIs, but at least at Shopify, we believe that will shift over time.
1. GraphQL is more powerful and you don't get all that for free.
Take a look at Apollo’s work on merging schemas together for a good example.[1]
Personally, I’m working on a proxy layer that will sit in front of a GraphQL API and act as a sort of business rule engine. It analyses the incoming queries to figure out which sets of rules need to be run, and then actually alters the incoming query to add in any additional data points required before forwarding the request to the backend API. Once the combined result comes back, it executes the appropriate rules and filters the output before sending back to the client.
These sorts of solutions don’t have anything to do with APIs. Heck, you could create a GraphQL schema to act as a classic data access layer, and call it internally from your own code.
2. You end up paying more attention to designing a good GraphQL schema/API because of the type system.
I’m not sure where GraphQL will end up either, but I can see a place for it just about anywhere. (I suppose I’m a bit of a kool-aid drinker.. I wasn’t when I first started using GraphQL, but it has grown on me)
[1] https://www.apollographql.com/docs/graphql-tools/schema-stit... [2] https://apis.guru/graphql-voyager/
It's easier said than done but I just think that there needs to be something for GraphQL that Rails was to REST, and for those options to exist outside of Node-land. I don't know what's out there for that, but I have heard a bit about absinthe: https://hexdocs.pm/absinthe/overview.html.
Having gone back to a project using REST and Redux, I miss working with graphql.
By graphql servers do you mean graphql clients on the server?
I'd think there's not much resources on building them because the basics are so trivial, the composition features are rarely necessary so all you need is the ability to POST a query, and parse the resulting JSON document.
My experience of interacting with the Github v4 API was understanding how to make queries:
requests.post('https://api.github.com/graphql', json={
'query': query_string,
'variables': my_vars,
})
And then some code afterwards to check for errors and pull out the bits into "native" objects, but that would be no different in an other API style.
https://jaxenter.com/graphql-good-no-alternative-rest-servic...
REST works in browsers because there's an intelligent agent driving the interaction. For an automated system it's just pointless drudgery and unnecessary extra requests and data. GraphQL basically fixes all the slow bits, by allowing for a single query in which the client specifies exactly what they want, it cuts down drastically on both the number of network round-trips and the size of data retrieved. And at the end of the day, that means it also (by and large) cuts down on server resources, because those extra connections and that extra data to extract & format is not free.
So GraphQL ends up being both more convenient and more efficient than "real REST services", and that's before we even get to the self-documentation from the schema and to graphiql explorers, which are superior to what "real REST services" could (but never did in my experience) (even less so as browsers kept on not supporting anything other than GET and POST) have provided.
Browsers seem to work fine, unless it's just been a tiny human inside my computer this entire time?
I'm guessing no, and that's exactly why REST work. The tiny human is not inside the computer, it's sitting between the keyboard and screen, able to (to a very variable extent) intelligently interpret every document's information and react on the fly depending on their whims and needs.
No, but that's not what rest means?
It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about, and it showed me actions I could click.
It kinda is, if there's no interaction back and forth the entire thing is completely trivial.
> It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about,
Little of which is of any use to two automated systems interacting without human intervention, and the bits which are of use are not hard.
> it showed me actions I could click.
That's the useful bit when working with an API, knowing that an action can be performed and being able to perform that action.
And that's where REST falls down, because at the end of the day it's not very helpful or convenient at programmatically exposing and allowing these interactions for automated systems.
Again, very good when there's an intelligent and flexible meat bag at one end of the interaction, but that's not what APIs are for in colloquial use.
GraphQL is (a DSL syntax for) a restricted subset of SQL—one that's restricted enough that it's okay to allow arbitrary clients from the public Internet to request of you.
Like SQL, though, its proper place is "inside" the API abstraction-layer, not "outside" it. GraphQL queries, like SQL queries, are things your backend app-layer logic can generate and send to your DB layer (presuming your DB layer supports GraphQL.)
There are certain APIs where you might want to expose the ability to use GraphQL to modify the composition of the tree of data-structures returned by a particular controller-route. But this doesn't mean you should be using GraphQL "as" your API. GraphQL here is just a parameterization of your API, like result-set pagination. The API itself still needs to exist regardless of GraphQL, and needs to speak something that allows for all the standard CRUD operations.
While a real-life application may want to expose a REST interface, there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
I didn't mean to imply the language syntaxes are related. But they're both syntaxes for relational algebra, and GraphQL has a strict subset of SQL's capabilities in terms of what relational-algebra queries/mutations it can express. (I.e. GraphQL can express joins, but only inner joins with an aggregate projection on the left side; and GraphQL can express projections, but only trivial projections applying the identity function to a subset of the available tuple fields.)
(Note that other, more powerful graph manipulation languages, e.g. Datalog or SPARQL, are also syntaxes for relational algebra. They're similarly limited in some ways compared to SQL—and yet the types of queries they express easily would balloon out to hundreds of lines of SQL.)
> there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
There is no inherent reason why a service cannot expose an SQL API, either, provided you lock down the restrictions on the role the DB is acting as (both in terms of ACLs on DML query types, and in terms of caps on compute time, memory, etc.)
But people don't do that, and they avoid it for more than just the challenging ops story it implies. You don't expose SQL-based APIs to the public internet, because you tend to build APIs to service the needs of particular clients with particular use-cases, and those use-cases can be optimized for by adding things like caching middleware, but only if you can filter out and tag the particular use-case each request is for. REST lets you do that; the use-cases are directly named by the (method + origin + path), and the semantics are already there, built into the protocol, for caching them, substituting them, access-controlling them, etc.
GraphQL, like SQL, doesn't have any place for naming the use-case of a request, in the request. So, if you want any of the useful properties like cacheability or access-control or whatever else to apply, you probably want your GraphQL request to appear "inside" a RESTful request, such that you still have individual REST endpoints for particular use-cases.
> GraphQL, like SQL, doesn't have any place for naming the use-case of a request
This is not true, see the docs:
> The operation name is a meaningful and explicit name for your operation. It can be very useful for debugging and server-side logging reasons
https://graphql.org/learn/queries/
You wrote...
> There is no inherent reason why a service cannot expose an SQL API
GraphQL let's you query across backends, such as relational (SQL), graph databases, REST APIs, it is backend agnostic. You're comparing apples to oranges, SQL & GraphQL are different abstractions.
Every modern DB labelled "NewSQL" is fundamentally just a "NoSQL" database that has been made to speak SQL. Sometimes it is, in fact, literally a previously-built NoSQL database, with an SQL adapter layer like https://en.wikipedia.org/wiki/Presto_(SQL_query_engine) used as middleware.
And your application can speak SQL, too! It's not that hard to:
1. grab an SQL2016.y grammar file and generate a parser for it in your language-of-choice;
2. expose a port on your daemon that speaks a protocol wire-compatible with some particular RDBMS's wire protocol, e.g. the https://www.postgresql.org/docs/current/static/protocol.html (like CockroachDB has done.)
If you do these two things, then other programs can connect to your app as a database, and use their ORM tools to handle your data model. (Consider this option if you're ever developing e.g. an MMORPG server; it can save hundreds of hours of work in exposing new APIs and deprecating old ones, for the cost of a little front-loaded work.)
---
But I digress.
You can make any application layer, or database layer, speak GraphQL. Because it's a syntax for relational algebra, and relational algebra is just "How You Describe Relationships When Making Requests." It has nothing to do with the fundamental underlying schema of the data; it's a language for communicating your intent—the things you want to get, or the things you want to change—by naming them through their relationships to other things.
Now replace "GraphQL" with "SQL" in the above and notice that nothing changes.
2 - forking databases to speak the same language still doesn't solve the issue of how to coalesce data from multiple backends into one response. You need an API gateway for that.
3 - forking a db would take months or years for me. I added graphQL to my stack in a few hours without modifying my backends, again because it's designed as an API gateway.
Yes they both declare data requirements. But no they aren't both relational and only one of these two technologies was designed as an API gateway.
> The API itself still needs to exist regardless of GraphQL
It really, absolutely, emphatically, does not.
For example, how do you implement your AAA login logic using GraphQL? You don't, right? Logging in isn't really anything to do with querying or mutating an object tree. (You can deform your logic into making it so, but you'll be breaking things like the ability to do OpenID Connect.)
How do you implement an endpoint to stream chat message events in GraphQL? You don't, right? You want a stream of trees? A tree of stream-poll-results? There's an impedance mismatch here.
You want to use GraphQL to upload attachments? Use GraphQL for webhooks? Use GraphQL for Operational Transformations in a Google Docs-alike? No, no, no.
GraphQL is useful for implementing the message-format of particular endpoints of an API. It doesn't work for all endpoints. Unless your API is entirely just about a particular object graph, you still need to put your GraphQL inside a larger abstraction that distinguishes the GraphQL endpoints from the other endpoints.
> You want a stream of trees?
I was under the impression that is what GraphQL subscriptions are for but I haven't had the chance to try them. Isn't that so?
> You want to use GraphQL to upload attachments?
I have been using apollo-upload and it works quite well.
Our solution is exactly like Stripe's. Clients ask for the relationships they want to fetch using the query string.
It's not ideal. Seems like a hack. But it works.
On the bright side, I can query the system using curl.
What should be my incentive to consider switching to GraphQL for this project?
What should be my incentive to consider GraphQL for my next project?
I'm currently writing the GraphQL API to replace our REST API. It is a lot of work, but when I have to put on my "front-end" hat it will be much easier to work with. I think the biggest payoff is for consumers.
- Lets you send only the information asked for. For example, you may be sending 50 fields when the client is only interested in 2.
- Lets you evolve the API more safely
- Enables better tooling for people who need to explore your API, like GraphiQL
The tradeoff is that I need to use GraphiQL or similar tooling to leverage this auto-complete, type-safe magic and that using curl becomes more painful.
I'm curious: Suppose you converted your REST endpoints into GraphQL endpoints in the simplest way possible by changing the query-params to root GraphQL nodes. So there's no "graph" really. Just lots of top level nodes. This reduces the API docs overhead and nothing else.
Would you switch? Is your existing tooling to extract API docs already good enough that this isn't a problem?
This is our API Documentation and it's generated automatically from our E2E tests. It's almost free for us to maintain.
Can you tell me if this is OK enough comparing to a GraphQL that wouldn't justify a rewrite?
[0]: https://facebook.github.io/graphql/October2016/#sec-Overview
URLs you can change by hand in API calls? When I'm at that level, I can just as easily edit the call body.
> One critique I haven't seen about GraphQL is that it's less human-readable.
My experience is the opposite, reading a complete graphql query is way easier than reading a mix of URL, query parameters and entity body, even more so when I need multiple such queries to replace a single graphql query.
And the query tells me exactly what's in the result, to boot, so it acts as documentation for the data-extraction code.
No, that's not what I meant. I meant that RESTful APIs relate the URL to the data in a way that isn't true in GraphQL APIs.
Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
Swapping one out behaves as I'd expect: https://www.reddit.com/user/faitswulff
And tacking on a format type to the end exposes the content in the manner I'd expect: https://www.reddit.com/user/masklinn.json
The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
If everything was an SPA, you'd have to interact with the data through the frontend, which speaks to some singular GraphQL endpoint, hence my mentioning that GraphQL itself mentions that it is "unapologetic" about its front-end orientation.
So 1. you're talking about something completely unrelated to TFA and 2. you're talking nonsense.
> Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
That's a UI component, and an explicit decision of the system's designers.
It, again, has nothing do do with APIs.
Hell, it doesn't even have anything to do with RESTful systems, https://www.reddit.com/lfskfsh8432kdb60 would be just as RESTful.
> The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
Cool, so use that for situations where direct human interactions is expected or convenient, what's the issue?
> If everything was an SPA, you'd have to interact with the data through the frontend
Which you already do, the URL is part of the frontend.
And using SPAs does not preclude providing nice (and user-friendly) URLs e.g. with https://developer.mozilla.org/en-US/docs/Web/API/History
There's no rules about how you have to use it, but the problem it was built to solve was communication between first-party applications and backends.
[0]: https://medium.com/airbnb-engineering/reconciling-graphql-an...
I've spent the past three years using Django Rest Framework. I switched over to using a GraphQL (Graphene as the provider, Apollo as the client) and it's been fantastic thus far. Easy to pick up, great documentation. Will learn more as we grow.
It translates GraphQL calls to gRPC queries.
How is that a good thing from the server’s point of view?
As mentioned elsewhere, almost no one talks avout the server shen talking about GraphQL.
When you can construct an ad-hoc query 4+ relations deep on the client, how can anyone efficiently handle this on the server? You can’t even properly cache GraphQL requests.
Initially we were just exposing a big Entity type in GraphQL which had a field for each component. Didn't feel like there was enough control over caching & prefetching. Current system is kind of nice because one reads/writes to redux, calls commit on ids to submit to server, querying just populates the redux store
The approach I recently implemented was to add a "type" or "code" property with standardized values to error objects in the GraphQL response (similar to error codes in Stripes REST API [0]). The client can then check those "errors" objects for specific codes and behave accordingly.
https://graphql.org/learn/serving-over-http/
> [a] query might result in some data and some errors, and those should be returned in a JSON object of the form {data: {}, errors: [] }
Also checkout https://github.com/kadirahq/graphql-errors which exposes a standard `path` field for each error, so the client has a standard way to know which part of the query resulted in an error.
If a mutation fails, for example a bank withdrawal fails because of insufficient funds, that is not necessarily an "error". To quote "code complete":
> Exceptions should be reserved for what's truly exceptional.
I'd send a mutation like this:
withdrawFunds($amount: 100, $currency: "USD") {
didSucceed
didOverdraft
canRetry
}
In the case you shove arbitrary error codes in strings, you lose the benefits graphQL brings, like tracking usage of a field, self documenting & introspection, solving for over/under fetching, and the ability to revise these codes without versioning your mutations.
You can even have a mutation save a nested object (fragmenting it and passing it down to other handlers), which is so great.
You can make a GraphQL endpoint quite fast with https://github.com/graphql/graphql-js
GraphQL queries have depth, when you call a toplevel endpoint which returns a non-trivial type you can select fields of the sub-resources to fetch, in fact you can also select just a slice of the top-level resource as well.
Let's say HN had a grpc API and you wanted the top comment of each article of the main page, unless it provided a very specific endpoint doing that you'd probably have to list the front page (and get a whole host of crap you don't care for e.g. the title, the destination URL, the score, the author, …), and for each article get its comments (again with all sorts of metadata you may not care for). In GraphQL, you'd select all but exactly what you need at once.
I'm guessing typical implementations would involve a lot of caching.
It would be nice to hear about people's experiences though.
What they do is coalesce all those separate queries into one, and actually allow for sub-query optimisations if you feel like it.
So the answer is that it does not, any more than e.g. grpc does not.
In some ways, analogous to your client using SQL instead of an ORM to query a system.
The only experience I have had calling GraphQL came from github and it was not great. The API was responsive but when I was trying to figure out syntax I was longing for a much clearer REST end point that would give me the data I was after.
What experiences have you had implementing GraphQL APIs, and what specific complexity differences did you observe?
What experiences have you had in maintaining living REST APIs used by evolving frontend clients, while balancing backwards compatibility concerns for supporting older clients? How has this compared to doing the same with a GraphQL API?
What experiences have you had in developing front-end web and/or mobile applications that consume a GraphQL API? How has that compared to consuming a REST API?
Have you invested any time in learning the GraphQL syntax and understanding the goals and intentions of the project, or did you just use it that one time with GitHub's API and get a bad taste because it was unfamiliar?
I will admit that for APIs provided for 3rd-party public consumption (like GitHub's), GraphQL is not as much of a natural fit for consumers (where, as long as GraphQL is not widely understood, it may just be a "figure out syntax" obstacle) as it can be for in-house APIs consumed by 1st-party front-end applications that utilize tooling built around GraphQL as an advantage. Nonetheless, I would be hesitant to advocate for REST over GraphQL in all cases (as you have) just because I had a bad experience consuming an API that used GraphQL when I was unfamiliar with it.
I'm a backend engineer and I maintain a living GraphQL API consumed by web and mobile engineers on my team. I also helped in our migration from REST to GraphQL, and it's been an overall positive experience for me and the rest of my team. If you stand by your statement that I am out of my mind, I hope you have some relevant experience that you can share to back it up. I doubt you do - your comment comes across more as vague new-paradigm-hate than as experience-based wisdom - but I would be happy to be wrong.