GraphQL Performance Monitoring Is Hard
medium.com
medium.com
A few years back I designed a REST API and made the mistake of allowing too many query parameters, which made it really hard to optimize the database as there were hundreds of possible filter combinations (some requiring joins) that a client could ask for and that needed to be covered with indexes, so while most of the common queries worked well some of the "long tail" queries would always run into timeouts.
When using GraphQL with a backend like Hasura that generates SQL queries I would think that you have the same problem for complex data models, as there will be countless combinations of filters and joins that your database would need to efficiently cover. From what I understand most organizations solve this by either restricting the types of queries you can make through GraphQL or by just defining timeouts and dropping queries that take too long (which does not offer a great user experience).
To all of you who use GraphQL in practice, how do you solve this issue?
It just converts a custom syntax represented in GraphQL into arbitrary SQL queries, which is absolutely absurd. They have mechanisms to restrict access, but the fundamental foundation of the idea is flawed.
GraphQL is great in that you define the exact queries that you know to be performant. Once you resolve your top-level payload, most of the nested fields will be non-parameterized relations. These are usually just tons of multi-gets.
So in practice, you have top-level queries that you know to be performant, followed by multi-gets that you know to be performant. If you do have a parameterized field, make sure it's fast.
How is this meaningfully different from having an SQL query fronted by a REST API?
I mean, what's interesting isn't what is possible, but what people actually do.
Normally, you have an API and 2-3 different clients. While the front-end devs define the queries while they are creating these clienst, once done, they will use these exact queries for most of the time, so you can optimize for them.
Think of an "advanced search" feature on the client that allows for creating queries of arbitrary complexity (basically, a client-side query builder). The user could certainly go overboard I'd imagine.
IMO, REST APIs are a better fit if you open APIs to the whole world. Then, at least, you have a limited set of queries that you can optimise and plan for.
GraphQL scares people because it seems to to allow arbitrary complex queries from applications. But that complexity was always there. REST just hides it in an unbounded number of predictable HTTP requests.
The same computatational challenges we've always had with mapping relational models to REST didn't go away, mainly: N+1 queries, recursive types and query parameters adding complexity.
With exceptions:
Preventing endless recursion: Or else it'd be possible for a client to timeout with queries [1]. In graphene recursion can be checked via `info.path`.
Calculated / slow fields: Makes more sense to have dedicated objects if calculations are involved, so if they're lists/connections, there can be limits imposed for how many can be grabbed at a time.
Sometimes the only way to be sure is to only allow access to these objects if they're a direct ID lookup at the root, e.g:
repos(id: <ID>) -> stats -> TotalLikesForAllIssues.
Assume that if cold and uncached, it could take <1s to calculate that. Imagine if we were to have that ran multiple times:
User -> repos(first: 20) -> stats -> TotalLikesForAllIssues.
So in my case I want to make the data available, but it's not always a direct mapping to SQL. And I'm also making rules inside where the client is limited in how, and how much they can query.
[1] Further reading: http://facebook.github.io/graphql/#sec-Fragment-spreads-must..., https://github.com/graphql/graphql-spec/issues/91
If your data models are pretty well structured, then basically every graphql object is a single table that you get by an indexed field, and traversing relationships can each do a where in query on the indexed id field. The parameters you expose on queries should be for indexed fields. So in practice, you’re not really autogenerating any particularly expensive queries, and if you assume the client isn’t asking for tons of data they don’t need, it’s not much different in query patterns than rest. Except that in some cases, the client can omit fields or related objects instead of re-using the rest route that’s a superset of what they need, so you can save resources sometimes too.
Then on top of all that, you have the option to not auto-map certain queries to the orm in special cases that are more complex, so you always can still write the code yourself to be more strict about what you allow. For writes, you typically would always do this to control exactly which fields you write.
I like the arguments in favour of GraphQL. I think that what's missing is the ability to set resource limits on queries or users. That's relatively common in relational databases. I saw it used by Oracle DBAs on a regular basis, I've seen it done in Greenplum.
Makes me wonder if Hasura would run on Greenplum, actually. Then you would essentially put backpressure on the client to send less intensive queries.
Disclosure: I work for Pivotal, which sponsors and sells Greenplum.
Similarly, Postgraphile (which I'm more familiar with than Hasura) allows setting either a max query cost or a max query depth [2]. I haven't had to use either, as I don't provide a public GraphQL API (and can just whitelist the queries I intend my clients to run), but there are ways to limit query complexity.
1 - https://developer.github.com/v4/guides/resource-limitations/
2 - https://www.graphile.org/postgraphile/production/#limiting-g...
GraphQL may one day have the tooling and ecosystem to reduce the net complexity of the entire system, but that day wasn't a year ago, its definitely not today, and judging by how long the community has had to mature and still hasn't... probably won't be for a very long time, if ever. Probably because, again, its pushed by frontend developers primarily, so the pace of innovation on the backend side of it is dramatically slower. Apollo Client is an amazing piece of software that does some insanely smart things; Apollo Server is a thin wrapper around Express which does nearly nothing of value beside "make a GraphQL server possible."
Given how slowly things are progressing in GraphQL-land, I'd estimate that it has another two years before the zeitgeist moves against it. Though I hope I'm wrong, because its got some pretty cool ideas and is dramatically better for frontend development than anything else out there.
[1] https://twitter.com/ryanflorence/status/1060557386421723136
/graphql/{queryname}/{hash}?{params}
Multiple advantages:
1) client doesn't need to send the whole query unless it doesn't exist. All of this is handled automatically.
2) http caching of queries are now possible since it's a get instead of post
3) we can check slow queries using similar tools as rest. When we see a query being slow, we check each resolvers. We usually keep the queries simple
I mean, not any more difficult than any other internal component of an application.
Yes, you can't just aggregate by URL endpoint, but add opentracing and start at the top level resolver, and then start a new segment at each nested resolver.
Poof - you have an extremely nuanced view of your application performance.
Add in aggregate time metrics for each unique query/mutation and you'll have an even better view of what going on - far better than if you were simply looking at API endpoint timings in the first place.
Yes, this does make instrumentation and rate limiting harder, but not impossible. It’s akin to using SQL... sometimes the query optimizer does weird things, but declaratively asking for your data is much better than imperatively pulling and joining it yourself.
Why? This is always my chief complaint about GraphQL: there is no reason it should take fewer trips than directly hitting an endpoint unless you're structuring everything very rest-y in which case you're doing it wrong and GraphQL is a bandaid (and you'd still probably make the same amount of trips unless you're heavily caching things). IMO.
> 2. Clients only download the data they need.
Most third party APIs let you specify what fields you want via a query string or post variable which amounts to the same thing.
Honestly I think if people stopped trying to force REST into everything and just made RPC like calls to HTTP we'd have far less of a need for GraphQL.
Don't get me wrong, I still think GraphQL is pretty cool for third party access in some respects but I don't think it's as useful as a lot of people make it out to be.
Not really. The type of API where you specify what you want and the API itself included whatever data you need in an RPC fashion has been around for decades, I feel weird calling it a "graphql api".
Ultimately GraphQL gives you some tools. The developer just has to figure out if those tools make up for the additional overhead (processing, size, training, etc). Sometimes it'll be justified I just think more often then not it really isn't.
You can roll your own version of the concept, but why bother when a perfectly serviceable one already exists? With already written client & server implementations in your language of choice?
GraphQL is indeed not a new invention. It's a standardisation of existing ideas, that has been given a particular name, and a whole community of people who are building tooling around it.
If you tell me you are using/implementing GraphQL then I immediately know what to expect. I know which tools I can use, I know you have a schema, I know I can run queries and the syntax of those etc. But if you tell me that you are using a REST/HATEOAS/RPC with some particular query syntax then I first have to spend a day reading through the documentation to understand the API.
i think you're giving graphql too much credit, and give too little to REST.
The schema of graphql is going to require just as much reading up and domain knowledge as the REST api.
The great thing here is that you don't have to anymore - you automatically support them all (or close to). And if it saves you from writing and documenting each of those new endpoints, that's a win at least in development time for me.
In most cases a thoughtfully designed REST API built according to HATEOAS shouldn't exhibit these problems. I've seen too many teams with a flawed understanding of both viewing GraphQL as some kind of silver bullet. Instead of stopping to consider their architecture they just rush out and implement GraphQL only to find themselves having significantly worse performance issues than they were having before.
user {
groups(first: 10) {
posts(first 1) {
content
author {
name
address {
state
}
}
}
}
}
/user
/user/12/groups?limit=10
/groups/34/posts?limit=1
/posts/56
/users/78
/users/78/address
/top_posts_view
I don't understand from the description "a thoughtfully designed REST API built according to HATEOAS" what your proposal to solve this problem is.
I'd have concerns about writing business logic in resolvers. If I was using it with a single SQL db I'd probably write a separate non-GraphQL layer for business logic and keep the resolvers thin.
For example, let's say you have have two endpoints /albums/1/ and /users/2/ and want to query them both at the same time.
They're unrelated data, so following REST best practices would make them separate endpoints.
With GraphQL, they're seperate resolvers but you can write
{
albums { get(id: 1) { .... } }
users { get(id: 2) { .... } }
}
GraphQL can do it, but I don't think that's a big strength, compared to the dependent queries it can do. Using the results from one query to drive a further query without going back to the client will save a bunch of time if the client is far from the server.
I do agree though with your point about dependent queries, though I have only used GraphQL for internal apis so my experience doesn’t include that sort of problem.
Unless you’re building one giant graphql enpoint exposing your whole database ?
Some trade-offs are things like more backend complexity, querying complex data (like deeply nested or recursive data structures) becomes harder, limiting exponential query complexity as the data branches out, etc. It also becomes more difficult to predict resource usage and to monitor performance, as the article mentions.
Most companies have denormalize data to a great many materialized views, to support multiple use cases. I don't see how GraphQL gives you those materialized views for free. I admit, I have not worked with GraphQL since 2017, and maybe the eco-system has come a long way. But at the time, we were doing all the work ourselves, and much of it was redundant with other efforts we had ongoing.
Most companies have denormalize data to a great many materialized views, to support multiple use cases. I don't see how GraphQL gives you those materialized views for free. I admit, I have not worked with GraphQL since 2017, and maybe the eco-system has come a long way. But at the time, we were doing all the work ourselves, and much of it was redundant with other efforts we had ongoing.
I'm usually surprised with all the comparisons to REST. It's far more practical and way less dogmatic.
I'd say that's because you should mentally substitute RPC whenever you see REST. Basically everyone talking about REST APIs mean RPC over HTTP with nouns in the endpoints.
It's the disadvantages, particularly the 'unknown unknowns' that I wonder about, and that's what's kept me from using it. GraphQL so far seems like the archetypal counter example for the 'use boring stuff' philosophy of software engineering. I kind of hope I'm wrong here, because the advantages are big and pretty cool.
It can get "exciting" on the client, particularly with libraries like Apollo. See for example the bug https://github.com/apollographql/apollo-client/issues/1186, which took two years to fix. The tooling for persisted queries can also be hard to set up, and they're close to required if you want to expose GraphQL to your client.
I mean what I mean exactly.
- GraphQL allows ad-hoc queries of unlimited complexity.
- Since it's POST requests, there's no caching on the HTTP layer.
- Since it's add-hoc queries, it's hard to devise a cache for the DB layer.
- There are no easy solutions for either authentication or authorisation for queries (and subqueries and/or fields)
> Because they are all solved problems
If by solved you mean "busily reinventing the wheel and adding layers and layers of unneeded complexity, and it will only work if you buy into a single vendor's solution such as Apollo".
For crying out loud, one of the "solutions" for caching is to actually parse both the incoming and the outgoing requests, read the fields and decide if it needs to be cached.
That sounds exactly the same as the thing you do when caching a HTTP request to me..?
Whether the fields are in the request body or the query string is pretty much irrelevant.
I think you are just annoyed that they misappropriated HTTP to do it.
ETag, Is-Modified-Since, Last-Modified, Cache-Control don't require any parsing of the request body.
And the relevance is huge. It's one thing to match a resource id in a URL in an idempotent cacheable request. It's another to have request body fully parsed by a caching middleware layer in a non-idempotent non-cacheable request.
> I think you are just annoyed that they misappropriated HTTP to do it.
They looked at HTTP and threw out everything HTTP already has built in. And all the tools.
As far as I understand it you can use the request body directly as a caching key. No need to parse it. If you request exactly the same thing there’s no need to re-request it.
Also, a quick look at the top search query for “Apollo graphql caching” shows that Apollo needs to deconsteuct the object (both for the request and the response) to be able to cache things: https://www.apollographql.com/docs/react/advanced/caching/
(I’m using Apollo here as one of the most advanced GraphQL solutions)
Only thing I'm currently aware of that might 'solve' arbitrary complexity and caching is limiting queries to pre-approved whitelisted ones the server knows about and references with an id, which is more of a workaround than a solution, and removes the main advantages for whole sets of usecases like open apis for 3rd party use.
- graphql() function takes "validators" argument for a reason ;) There are multiple strategies for limiting complexity, but common ones are boxing (limiting depth and width of query) or query cost limitation.
For auth, you authenticate user using custom middleware before entering the query executor. Inside query executor you can do one of many things, depending on what makes sense for you: error field resolution, return "null", or return custom type like "ObjectNotFoundError".
For caching, you cache inside resolvers that do something costful, or in data loaders those resolvers call.
Are things like query depth, width and cost limiting available as parameters to common libraries?
On the auth thing, one thing I commonly do with REST is have a general 'is this user authenticated' middleware and then later have a 'permissions' middleware that checks if the user has rights to see/modify the thing they are trying to, but that's at the service level for that particular thing.
Would you follow a similar strategy in GraphQL? Generic auth in a middleware for a user, then specific access management auth inside specific resolvers later on?
No, they are mostly 3rd party utils. I those exist for Apollo server.
> Would you follow a similar strategy in GraphQL? Generic auth in a middleware for a user, then specific access management auth inside specific resolvers later on?
Yes. Or I would move auth check up a level, and implement schema directive like "@authenticated" or "@hasPerm("perm")" so resolver would do only data resolution, but each approach has advantages and downsides.
Caching is no harder in GraphQL than it is anywhere else. You define resolvers; if you want to put stuff in Redis, you're free to do so.
Auth is also no harder in GraphQL than it is anywhere else. You have complete control over what happens in a resolver, which includes authorising access and throwing an error/returning null/etc.
It seems to just move all the flexibility of GraphQL to dev time, and makes it only useful for a private API where you control both the client and the server. This is all fine, but then I just don't see why you'd move away from a classic API where caching is free for GETs and you don't have to go through the abstraction of GraphQL schemas and resolvers etc to do your endpoint logic, you can just do it.
One thing I could possibly see is that client side libraries like Apollo etc bring huge advantages like free client caching or other things I'm not aware of - is that the case?
Apollo was really useful for managing state across the client. Tools like optimistic updates were very nice too. Unfortunately, it was somewhat buggy when we were using it, and the API kept changing, but I think it's more stable now.
It was also good to have a well-defined schema between client and backend, although you should have one anyway.
I've worked for companies in the past that had the organisational setup you describe, however, where frontend engineers didn't even have pull access to the code for api services they were talking to, where this would have been a massive advantage in terms of productivity.
I seem to end up at this conclusion in most discussions about GraphQL, usually with facebook scale being referenced, where it seems that as a technology its main advantages might actually be architectural in terms of the organisation of how massive systems are engineered, by huge numbers of people.
The temptation is always to jump in and compare the technical merits and specifics, and overlook this, but I'm coming more to the opinion that this is the answer here to when to use GraphQL over REST / RPC over http. That is, at the scale of a massive engineering organisation, it might be an abstraction whose advantages outweigh the downsides, but probably not if you're a small team.
If you're finding that your endpoints are filled with unnecessary optional fields, that they're specific to a single screen in the UI, that nearly every new feature involves work that's duplicated across front-end and backend, or that you're suffering a performance hit from the round-trip time between front-end and backend when you query nested objects, then GraphQL can be an option. These are problems that you run into when your product starts to get bigger and more unwieldy, so if your UI has five screens and doesn't reuse the same content then you might not see a lot of gain. Having said that, my former company was under ten engineers, so I don't think you need to be Facebook-scale to benefit.
Totally agree that the advantage is architectural. It's not a question of technical merits or of REST being bad. GraphQL is a layer of abstraction to separate data from presentation, and like other abstractions, you shouldn't use it unless you have a reason to.
> The solution to ad-hoc queries is to not use ad-hoc queries. Persisted queries are becoming common.
So, you either reinvent REST, or fall prey to ad-hoc queries with potentially unlimited complexity.
Also, persistent queries don't save you from unlimited complexity when those are automatically persisted, which is what Apollo offers.
> Caching is no harder in GraphQL than it is anywhere else. You define resolvers; if you want to put stuff in Redis, you're free to do so.
Putting stuff in Redis does not make it caching.
> You have complete control over what happens in a resolver, which includes authorising access and throwing an error/returning null/etc.
So, you throw an error in a deeply nested resolver in an ad-hoc query with unknown complexity, and .... this is "no harder"?