GraphQL and the Beads on a String
blog.luccasiau.com
blog.luccasiau.com
Once I ship I plan to put together a nice sample repo talking about why I like it, but you can see the WIP here of how I use WG to build a system with (potentially) multiple databases being consumed in a type safe manner from my ios app, nextjs app (and anything else)
What happens if I say it three times in a row?
Will it summon you to appear in my geographical proximity?
WunderGraph WunderGraph WunderGraaaaaaAAAAHHH-
https://twitter.com/meixnertobias/status/1704565814022770857
1. https://github.com/manifold-systems/manifold/tree/master/man...
This seems like such a vastly overcomplicated way of doing things that I cannot fathom it. Throw GQL in the mix (which is already itself a thick abstraction layer), and it overwhelms me. The number of implicit failure cases, alone, makes my head spin.
Maybe I'm old school, but my solution is to write a bespoke version of what the kids call a "back end for front end" that calls the APIs, handles the error cases, and consolidates the data into a single REST endpoint built-to-purpose. You don't need GQL. You don't need a library. It makes your life less complicated -- you have a single endpoint to test, there's no leakage of the back end into the front end, and it's easy to get things (like, say, logging and analytics) without depending on expensive JS proxies such as Segment.
The only difficulty is that your back end engineers and your front end engineers have to actually talk to each other and coordinate (to be fair, this seems to be the difficulty driving 95% of this stuff: the front end engineers want to make arbitrary changes without depending on coordinating changes in the back-end API.)
Either you will have complexity maintaining a bunch of endpoints, keeping track of where each one is used, what is returned, where the contracts are defined, writing tests to make sure nothing breaks, .... Sure, you might not have much abstraction and it will be cleaner that way, but it will be a nightmare to maintain, especially in the phase when you are moving fast and breaking things as you find market fit
Or you can rely on heavy abstraction and code generation with end to end type safety so if you change something you instantly know if you broke anything at compile/deploy time (or earlier).
Something about building this way is really nice, and quite different than anything else. Personally, i'm betting big on code gen. If coding behind layers of abstractions make you uncomfortable, whats coming down the pipeline is going to be giving you nightmares (https://twitter.com/hrishioa/status/1748346491528532344)
Yeah, I hear this argument a lot, but it hasn't been a problem in my lived experience. Certainly for small teams (and absolutely for apps written by one person), it's not that hard to keep track of the endpoints. Even for larger orgs, teams tended to want to keep their endpoint counts low, and would push back on the sorts of expansions in back-end complexity that these frameworks (i.e. GQL) make "easy" (but in reality, just kick the technical complexity can down the road; it's a lot easier to see the horrible DB queries when they're not invented dynamically by the one GQL query you didn't think of, written by some Future Intern who doesn't know any better).
I can see how this becomes more of a concern when you've got a massive org that has doubled down on microservices, but that's not most orgs.
I'd much rather if I make a change to an endpoint, my project literally won't build and my IDE yells at me. I can make sweeping changes and go to sleep knowing I didn't mess anything up. And I know if I bring another engineer on, they can make sweeping (or tiny) changes and not have to know all of the places that consume that.
I should also note that WG doesn't require graphql. I have a few graphql operations, but you could use it completely without any graphql operations.
I think you're making an allusion to GQL typing here, which I grant. A big downside of REST+JSON is that you've pushed most of the "interface contract" to the structure of the JSON file.
> I should also note that WG doesn't require graphql.
Sure. The common element to both (for me) is the layering of abstractions that hide what's really going on.
Ya, for rest and json that was my challenge. I liked the idea of openapi, but manually creating openapi docs was just not much fun.
If you are really curious about WG approach this is a good intro: https://www.youtube.com/watch?v=m3YrZav5-CU. They have some contrarian takes within the graphql community
The second I wasn't part of when they made the decision, and they decided to use GraphQL and now regret it. It works, but it makes things very complicated. We're a single large team responsible for both the front and back end (in fact, two front ends).
GraphQL seems mostly important when front and back end have separate teams (possibly even at different companies) and the front end needs very specific custom data from the back end.
Additionally; GraphQL never got the community love it needed on the backend side to make things hum. Apollo is the only organization really pushing it, and they invest something I'd estimate as 10% of their time into thinking about Apollo Server, which is in any event JavaScript-only. It turns out, the problems GraphQL creates in a system are backend problems; there's some really freakin hairy edges in the domains of authorization, caching, rate limiting/complexity limiting, even just basic type safety, that can only charitably be described as "solved" if you consider "some ideas in a twenty page blog post" as a solution.
We also had constant problems educating our customers about GraphQL. "No no, you can use any REST client you want, here let me show you how to set it up" (two hour education session later).
There really isn't a perfect solution to the problem space. If your view is complex enough, synthesizing it JIT in GraphQL is just the frontend saying "its your problem now" to the backend, and whether the solution to that is "fixing performance bottlenecks" or "a new RESTful-ish view API for this screen" for the backend team, its still a ticket. If the view is less complex than that: making multiple requests isn't the end of the world. After all, those two highly predictable, optimized, even cached requests may end up being faster than one dynamic and extremely-difficult-to-cache GraphQL request.
At the end of the day, I think there's a reason why REST has lasted as long as it has, and why it still feels state of the art to me. Moving more API capabilities to edge platforms, including edge databases, shortens that string of beads quite substantially. Server-side rendering is back is in full force. There's other angles this problem is being attacked from.
Ok, well for some reason every project I've been on in the past 4-5 years has used GraphQL? What makes me so special?
As I conclude that I am not special I have to go back to what my previous position was that GraphQL is pretty much the standard way front-enders query nowadays.
That said - the project I am on right now only uses GraphQL for querying CommerceTools, and their GraphQL implementation seems like someone told a dev hey, we need GraphQL, make our SQL backend available over that, and he completed his ticket anywhere from 4 hours to 4 days, but certainly not in any amount of time to make a solution that had any of the advantages of GraphQL in it.
We used to have GraphQL for the part of the project using Contentful - but their GraphQL was so problematic we moved to their Rest API (I say was because that move was over 1 year ago, it would be wrong to say it is bad now - who knows what improvements they might have had)
Aww, don’t sell yourself short!
Even Meta has a "Graph API" that is not GraphQL (https://developers.facebook.com/docs/graph-api/overview) (they may have an odd GraphQL API here and there; I am not deeply familiar with the entirety of their offering, and I don't want to insinuate such; just that they don't seem to rave about it front and center if they do, and instead invented a second, separate thing).
I interact with a GraphQL API among much smaller integration partners maybe once or twice a year; versus dozens of REST APIs. They're definitely around, people definitely still use it (my company has one!), but arguing that its the standard way frontend APIs are written feels very detached from reality; and maybe you really are special.
The Guild [0] does some exceptional work in the GQL world and specifically server side with Yoga, Envelope, Hive, and some other stuff; but it does tend to be JS focused for now.
Just make another endpoint for getting the whole thing in one go and call it a day, don't try to overly generalise your API.
Going even further, your server requests to the database could be similar as well, one query to get everything needed, then you don't even need to worry about server to database distance!
[1] https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
Would you elaborate on "queries are append only"? I fail to understand the downside you mention.
If you mean that you can only append to the data you return from a resource, you can use versioning to deprecate properties on a resource.
Maybe but the implementation is orders of magnitude more complex compared to simply making a couple of DB queries and then sending a JSON.
Don't use tools unless you benefit from them. Some people benefit from being able to define JSON responses in a graph query language.
It's easy to say "don't do that" but shit happens: large legacy codebases, social factors, and individual incentives of backend and frontend teams with deadlines. GraphQL makes the outcomes of these situations a lot more manageable. In that way I think of it as a social and organizational technology as much as a query technology.
> It's easy to say "don't do that" but shit happens: large legacy codebases, social factors, and individual incentives of backend and frontend teams with deadlines. GraphQL makes the outcomes of these situations a lot more manageable. In that way I think of it as a social and organizational technology as much as a query technology.
I think people miss the point a lot when ranting about graphql. It’s not particularly their fault, I didn’t get it either until I experienced multiple massive rest messes like you describe.
Also, as social/org tech you don’t really see the benefits in small teams / simple products / small apis. It just feels like extra complexity at that size.
I don't buy this argument. Imagine if an airplane manufacturer had the same philosophy. "We can't just substitute that component because it wouldn't fit the existing frame... Oh but wait, we can't just change the frame or the new component to make it fit because of social factors within the engineering team. Let's just tack it on as originally planned and then work around the issue by implementing a complex solution in the software to make up for the structural design flaw.."
That's basically the Boeing 737 MAX story. We know the result of that philosophy.
That's pretty much the point (well, one of the main points) of GraphQL... precisely so you don't have to do that. Want to request a new object or list of objects? Request a new field on an existing object? You don't have to go "make another endpoint", you just change your GraphQL query.
This is really nice for iterating, though it does have some downsides (eg it's easy for the person iterating away on the FE to not have to think about the cost of querying that data on the BE).
GraphQL does have a few other advantages though, like being able to build your typing around it (again, doable with REST but with more work).
I'm not actually opposed to Graphql as a technology, One of my favourite pieces of technology is actually PostGraphile.
I just believe it's easy to misuse it (in similar ways to an ORM) that result in not utilising server capabilities to their fullest.
And puts all the complexity on the backend :-)
As far as I've seen, front-end developers are generally quite happy about graphql. Back-end developers on the other hand are a different story.
So if GraphQL front-end developers into discussion about schema I rather take that over whatever REST solution they're using today.
But I might be biased here...
So basically frontend+backend effort with GraphQL is much smaller than frontend+backend with REST
Exposing a GraphQL endpoint has the same issues as exposing a DB. You need to define granularity of access to each individual table, column and row for each user.
I’ve yet to find a GraphQL interface that doesn’t expose too much information or fail to follow the principle of least privilege. Even IKEA, who I think has possibly the best GraphQL corporate-built implementation I’ve ever seen, exposes a little too much.
Think about your IAM environment and all the exceptions you need to grant users. The same thing exists for data and eventually the ease of use we trade for developers to rapidly prototype results a data breach. Why? Because someone enabled introspection to debug a production issue, or forgot to tick a box or define a parameter when deploying a data model change. Boring old RPC and REST typically doesn’t carry this level of risk, or has tooling to prevent it.
As someone who chases bug bounties, please keep building out GraphQL. My mortgage appreciates it.
But as someone whose day job is more on the blue-team side - no, you won’t get my sign-off. This is not acceptable risk.
The other aspect is with REST the interface is strictly defined. There’s no unknown data being disclosed, you’ve defined the resource and understand the authorization required by building it in the first place. Yes, mistakes are made by engineers who don’t understand what PII is or don’t care what the access control for the endpoint looks like, but at least you can spend time actually scoping it and defining what that looks like.
How do you threat model a constantly changing database? You could throw flags on fields and have that tie in with your GQL orchestration lib/tooling, but that’s something that very, very few organisations can get right.
Not even 1%, I’d say maybe less than 100 in the world.
I don't get it, how is this significantly more complex than what the author suggests? How is this shifting complexity to the backend with REST and not with GraphQL if all you're changing is the transport layer?
Like one time, I was working on a social kind of app. I thought, lemme give a separate endpoint to get each kind of object (user, post, etc) so the client can cache stuff. Frontend devs instead wanted everything needed to render each page in one call, which is understandable. I gave them that, but often they'd later need something I wasn't sending back yet, which slowed down development. So I started sending extra fields back just in case, e.g. sending a full public user object for some page that only needs the name and avatar. In short, it got wasteful, but not the end of the world. GraphQL would've made it cleaner, but it also had an upfront cost.
Single request and response and with a graph of nested relationships. This approach lets one still use HTTP GET caching, whereas GraphQL hides everything behind a single POST endpoint.
when splitting your request the user can view his profile picture before the main content finished loading. this cam hurt user experience
Server side:
https://the-guild.dev/graphql/yoga-server/docs/features/defe...
https://www.apollographql.com/docs/router/executing-operatio...
Client side: