Solving the double (quintuple) declaration Problem in GraphQL Applications | Hacker News Reader

Solving the double (quintuple) declaration Problem in GraphQL Applications

wundergraph.com

73 pointsjensneuse45 comments

> Most Web Applications are just forms that talk to a database.

And this is why the author is frustrated - GraphQL is designed for larger, more complicated systems. It's overkill for a basic web app like this. In fact, coupling the different declarations together like this defeats the purpose of GraphQL.

One of GraphQL's purposes is to let the different layers and parts of a system evolve independently. For instance, the database schema can be changed without changing the GraphQL schema. A resolver can be changed to pull data from a new service instead of the old monolith. It also affords more custom implementations, where the db schema is not exposed directly to the client or the data is stored across multiple technologies.

There's a larger point here, which is that it's okay to have multiple definitions for things that seem similar. It's okay to have one version of a User for the front-end, a different one in the GraphQL layer, and several different ones with varying details in different backend services. The key is that these Users are different things - for each context, there is a useful definition of User that is different from other contexts [1]. Some systems need security details, some need contact info, some only care about the user id, etc.

It's really important in large engineering orgs to allow these different definitions to vary independently so that teams can make incremental changes without updating every single other system. Hence tools like GraphQL.

[1] https://martinfowler.com/bliki/BoundedContext.html

Hallelujah. I see all these posts about auto generating GraphQL from your DB schema and I always think "But you're literally getting rid one one of the best parts of GraphQL by tightly coupling your DB to your API". I also feel that this approach is putting people in for a world of hurt: it's easy to get up and running quickly, but it will kill productivity later on when it makes refactoring much more difficult.

FWIW I define my API using the GraphQL schema language and then generate everything after that (Typescript definitions for server and client) using graphql-codegen and it works great.

I find these DB-to-REST, DB-to-GraphQL and so on variety frameworks to be terrible.

That said I've come to accept some people really just need to expose their database out there. And so having these tools help them.

It would be nice if those tools did provide for a smooth path from 1:1 mapping to full customization. Unfortunately, most don't.

I completely agree with you. I've also wrote this post explaining why tightly coupling the DB to the frontend is such a big problem: https://wundergraph.com/blog/tight_coupling_as_a_service

That said, I've added support for direct DB access to WunderGraph based on demand from the community. Although it has drawbacks, users want to expose their DB as an API. I don't want to ignore them.

That said, I also want to address your second point. WunderGraph offers two abstraction points. For one, you can always swap out a backend and join multiple APIs together. This means, you could replace an API generated from a DB at any time, even partially, without breaking the client contract. Then, there's also a second abstraction point which has to do with the fact that WunderGraph persists GraphQL Operations by default. It allows you to swap out the entire schema if the Operations stay the same. Both options might not always work but offer good migration paths from a DB generated API away. If you have the resources, it always makes sense to "design" your API. Generating an API is not wrong in many ways. However, for prototyping and simple projects, it might be the right tool to get the job done in a cost sensitive way.

To sum it up, generated APIs are not ideal but it really depends on the case. In the end, the customer decides.

I agreed with this until I found Hasura.

For me its just the right level of abstraction and saves me a boat load of time writing CRUD queries. Now, 90% of my CRUD ops are autogenerated, with great, granular authorization, and for more complex logic I can easily "join" a lambda function.

GraphQL is extremely badly designed - try to do nested pagination, that's basically unsolvable in a single server roundtrip. The hack of keeping an n-dimensional array of pointers to cover each level of pagination falls apart with any concurrent changes. And nested pagination pops up often when doing advanced UI on mobile on web.

The "nested pagination with a single server roundtrip" requirement seems a little odd, I'm not sure why that's necessary or desired. I can imagine an example, a view with a list of users, then expanding one user inline to view a list of their comments. But I don't see why I'd even want this whole nested view to be returned in one round trip, it seems quite wasteful to return the first page of comments for every user before it's expanded. Do you have a better example?

The need for hitting nested “paged” data structures on first render isn’t that contrived, but I’ve never found myself needing to juggle the cursors, since you’re usually only hitting the first page for the nested ones. Loading subsequent pages would be a standalone query, rather than running the root query again with different cursors.

(the example I came up with for this article a few years has a fairly realistic example of nested pages: https://andrewingram.net/posts/optimising-your-graphql-reque...)

What does an ideal solution look like in your view? Do snapshot queries help, so you're protected from concurrent changes?

With fquery (https://adsharma.github.io/fquery/), the problem boils down to constructing a new query with additional where clauses.

Because of the property that the shape of the query and the shape of the output are identical, nested queries fit naturally into the system, as opposed to GraphQL <-> SQL where one is nested, but the other is flat, requiring a mapping.

Efficiency often takes a back seat to Conway's Law.

REST has the same issue.

Yeah. The unacknowledged assumption being made is that n-definition (much less of a mournful) is inherently a problem. The reality is that each layer of the stack, the definition means a different thing even if the fields are the same _right now_. I’m an advocate of utilities to flatten the layers as an effort-saving measure, but as something to opt into, rather than out of. The layers must always be conceptually there.

The reason I don’t like things like Hasura et al is because they flatten by default (your GraphQL server essentially becomes just an ORM for your database), and you’re made to do extra work if you don’t want that.

I've dealt with the same issues in nearly every non-GraphQL project I've worked on. If you use types, you need to define those somewhere that each system can understand.

In most REST/JSON/whatever, I've worked with this ends up as serializers and deserializers on both ends of the client/server. At it's most basic, you're doing no more work in GraphQL than a "legacy" client-server.

-----

I've dabbled with GraphQL in personal projects and I think it far exceeds anything I've worked with in a REST setup. The major issue that I see with GraphQL is it lets you do stupid things incredibly easily. However, this is also it's benefit - a client can grab only and exactly what it needs.

The major problems I see with GraphQL is developers using fundamentally poor relational data models.

While I'm very excited about the emergence of type checking in "modern" web development the triad of GraphQL / Typescript Types / JSON-Schema is a real cluster and I hope we engineer ourselves beyond a solution that requires endlessly translating between them in the next few years.

After having iterated through a bunch of permutations of Rust frameworks, Typescript frameworks and ORMs here is a stack that I'm particularly excited about (that mirrors this article a fair bit):

- Design a Postgres database in whatever means you prefer (I prefer SQL, since it's easy to design constraints in using the full power of the DB engine).

- Use Postgraphile to auto-derive a CRUD GraphQL Schema for your database (this is similar to Hasura, but encourages using Postgres row-level security if you need fine grained access control).

- (Also) Use pgTyped for extensions to Postgraphile's schema when you want to implement mutations outside of DB functions.

- Use Apollo GraphQL Codegen in your front-end to auto-generate types for your GraphQL Queries.

The net effect of this is you get full type safety and your DB Schema is your singular source of truth (Note that I'm not auto-generating forms from JSON-Schema like this post, but this isn't necessary for our particular use case).

The way I see it, we got here by abandoning OOP for web development. There’s a lot of reasons that MEAN, NoSQL, no-schema… is so appealing, because OOP is so tedious. But we’ve just been slowly recreating OOP in a piecemeal fashion, with ORMs, TypeScript, JSON Schema, GraphQL… because the new approach didn’t actually provide an alternative solution to the problems that OOP attempts to address, it was just based on the premise that you could ignore them.

I don’t think we’re that far away from coming full circle back to OOP, even if the “framework” ends up having a different name.

I'm not sure I follow, what is the "OOP" solution as you imagine it here (that all these technologies are alternative to)?

The OOP approach is to define your entire data model as a collection of classes and interfaces before you write a line of business logic. This approach is designed to support a systemic methodology for modifying and extending the codebase. But it also adds a complexity that requires large quantities of boilerplate to be written, and requires maintainers to navigate through several layers of abstraction if they’re trying to read or modify the code.

From my perspective, I see the rise of Node and Python for web dev as a rejection of OOP, largely on the basis that web apps don’t need such a complicated set of structural abstractions. However if you look at how that domain has been innovating, my argument is that they’ve simply recreated all of the old OOP abstractions with different names. If you look at the types of abstractions used, and their associated complexity, in a typical GraphQL + TypeScript application, how are they substantially different from full OOP C# app for example? Especially if you’re using an RDBMS, or ORM, or some sort of schema for a NoSQL DB. For devs that don’t like TypeScript, JSON Schema might be appealing. But if you look at the JSON Schema spec on GitHub, one issue you see repeatedly raised is “please add full inheritance”, followed by a debate around “but that’s just OOP”. The community’s raised that issue so many times that I think they’re actually considering it now.

I like to start with code. If you use typeorm and type-graphql you can write a Type once and generate the database as well as the resolvers out of it. You can then even import the same type in frontend (as part of a mono repo). This stack does not need any generation routine for types. You just have to generate a migration if the postgres target schema changes, but that's normal and I don't see a way around this for the future.

SQL is code too :), but I follow your point.

Typeorm looks nice, but I need polymorphism for some of our "business" logic and have pretty specific requirements about how I want correctness enforced that typeorm doesn't appear to support (but raw SQL does).

Typeorm does not scale to large datasets. Real nice for MVPs. Just a warning. Hydration times explode for lists and joins that postgres barely registers on the query side, and hydration is blocking.

Money quote:

> "Most Web Applications are just forms that talk to a database."

Yeah... so why are we doing all this crazy stuff again?

Maybe we could just use hypermedia for this relatively simple problem? Sure, there are usability issues at times, but there are solutions to that [1][2][3].

I read stuff like this I go zoolander[4].

---

[1] - https://unpoly.com/

[2] - https://hotwired.dev/

[3] - https://htmx.org (my solution)

[4] - https://www.youtube.com/watch?v=D4f2j-Xmgc8

Maybe you're right, I don't know. What do you think why are technologies like React, GraphQL and NextJS growing their communities while hypermedia seems to be a niche thing? These frameworks are bloated and add a lot of potentially unnecessary JavaScript to we apps. At the same time developers seem to like the developer experience and trade this with a better technology like e.g. hypermedia APIs. Is it the usability? Lack of tools? Your tool seems to be thought out well. What's preventing it from becoming mainstream?

it's a complex story, but here are some reasons:

- HTML by itself had usability issues that were addressable only with javascript

- REST/HATEOAS got lost in the morass of JSON APIs, which isn't a natural hypermedia, and none of the thought leaders came out strongly for hypermedia or even really pointed out what was going wrong

- Developers tend to prefer RPC over Hypermedia because it is closer to their natural way of thinking

- FAANG-chasing

- Developers generally don't like saying "this is too complicated" because it sounds close to "I am not smart enough."

Hypermedia is too good an idea to die completely, and I think we'll see a swing back. But in the meantime, a huge amount of time and energy has been and is going to continue to be wasted on rube goldberg contraptions to make "forms talk to databases".

I’ve implemented hypermedia APIs before. It was a lot of work for exactly zero benefit because none of the client developers wanted what they were given in those APIs, and we were forced to develop _additional_ APIs that gave the client developers something they felt comfortable working with.

Were the hypermedia APIs returning HTML or some other real hypermedia? I assume not.

Were they consumed by a hypermedia client? I assume not, since there is nothing to complain about except compliance with the hypermedia specification in question.

Hypermedia APIs never made sense once we flipped to JSON apis, it was all cargo cult. I've written some essays on this:

https://intercoolerjs.org/2016/01/18/rescuing-rest.html

https://intercoolerjs.org/2016/05/08/hatoeas-is-for-humans.h...

Your premise about “HTML or some other real hypermedia” is bunkum. There is nothing about REST or HATEOAS that depends on HTML at all. The whole _point_ of REST was to build on the semantics of HTTP without regard for the transport protocol or encoding format (and HTML, XML, and JSON are all encoding formats).

What makes a RESTful API a HATEOAS API is that it provides you actionable hypermedia definitions. There is fundamentally no difference between:

    <link rel="self" href="https://my.server.example/api/customer/id/1">

and

    { "link": { "self": "https://my.server.example/api/customer/id/1" } }

When you include child responses, you include reachable endpoints for them, too.

    {
      "link": { /* top-level item refs */ },
      "posts": {
        "link": { /* post collection level refs */ },
        "items": [
          { "link": { /* post level refs */ }, /* the rest of the object */ },
          /* more objects */
        ]
      }
    }

And yes, in fact, that is _exactly_ what the API that I developed did—so every application client that worked against the API was required to work based on that (and it made making Postman collections really interesting, because you could not just “guess” at the URL required, you had to look it up through the chain). It worked.

While it felt nicer from a “purity” standpoint, it made the server slower, less agile to client requirements. We eventually abandoned it as a mistake in the next generation platform that we built.

There are a lot of good things about REST, and I like the idea of having discoverable APIs as expected in HATEOAS. But the number of people who _get_ it on the client development side is vanishingly small, and there’s always unfortunate drift where you get people templating API URLs even though they are given the exact API URL required to perform an operation.

These days, I’d much rather write up a good GraphQL API with Absinthe and just get the job done rather.

No, it isn't, and the fact that you found it to be useless in a JSON API is evidence for my claim.

The problem here isn't the API you designed per se: sure you can encode a hypermedia on top of anything, even JSON. The problem is the clients aren't hypermedia clients and don't want to interact with the API in that manner. They aren't using hypermedia as the application of engine state, they aren't taking advantage of the uniform interface and all the rest of it. If you read "HATOEAS is for Humans" I try to explain why that is.

All the problems you are pointing out are due to the fact that your hypermedia-on-top-of-JSON API is being consumed by non-hypermedia clients who, at the end of the day, just want a powerful RPC/Query mechanism. Which is fine, I'm glad you came to your senses and gave that to them.

But, to return to the original point, if you are just trying to get forms into databases, hypermedia is much simpler.

I read your articles. I disagree with your foundational premise. HTML is a presentation / structure mechanism. HATEOAS isn’t _only_ for humans, but HTML is.

The APIs that I built _were_ hypermedia. The only way to properly use them was to follow the links in the data. The problem wasn’t the APIs or even the software clients. It was the developers who didn’t see the point of it and ended up developing versions that broke the pattern.

If all you have to worry about is a web browser, then you can start from your foundational premise. Most of us have to deal with all sorts of clients.

> It was the developers who didn’t see the point of it and ended up developing versions that broke the pattern.

lol, cool man, take a week or two to think about that and then maybe reread my article

:beers:

Complication leads to higher cost of ownership, including training the junior devs who inevitably wind up maintaining the thing

I don’t care about hypermedia in particular, but it is intriguing how one can say that “apps are just forms” and justify all that tooling. Might as well just go with CouchDB.

GraphQL simplified the frontend by making the backend 100 times more complicated. So many fewer requests, though! Problem solved. Big names made and books sold.

> GraphQL simplified the frontend by making the backend 100 times more complicated. So many fewer requests, though! Problem solved. Big names made and books sold.

As opposed to making the backend simple, by making the frontend complicated?

GraphQL is SOAP 2.0

I already solved this problem for me. I use graphql code generator (https://www.graphql-code-generator.com/) to generate graphql resolvers on BE and angular services on FE. I use MongoDB and database models are written in typescript. I want to manage db models separately from the API. With this approach, I'm type-safe from end to end.

Edit: I forgot mobile. Apollo Android client generates schemas from schema and queries. I've never written iOS app, so I can't recommend anything yet.

Same. I paired that with https://typegraphql.com/ (which builds the graphql schema dynamically from the code using buildSchemaSync) and MikroORM (which you can overlap your entities with to remove duplication there) and it was a totally generated solution with full type safety.

A bit crazy complex to setup so many moving pieces, but once it is, it works great.

I had some issues with typegraphql years ago. I tried to split resolvers between modules and it behaved weirdly during unit tests. We also already used graphql-compose at job, so I wrote my own typescript decorator based solution on top of graphql-compose (https://github.com/captain-refactor/graphql-compose-typescri...) My module worked as expected, and I tried it on my personal projects. Then I realized, how my projects code became confusing and result api was really bad. I abandoned development of the module, because it had flawed design from the start. The code-first approach doesn't really work well and result graphql schema is ugly, which makes it a bad solution. Generating typescript from schema gives better results. When you think about it, it makes sense, because your graphql schema is your applications api blueprint and it should be designed in advance, not generated from the result product. Also when I wrote idiomatic typescript code, it generated bad graphql schema structure. It's hard to create consistent api when it's scattered over many files filled with implementation details.

It’s been interesting to see the ecosystem around GraphQL experience the same problems and finally reach parity with oData.

Current stack for creating easy to use GraphQL APIs + Frontend Integration:

- Django 3 backed with a Postgres database (standard stuff) [0]

- Add Graphene to enable Relay [1]

- Setup a fresh TypeScript based NextJS project [2]

- Add graphql-code-generator with a few plugins, mainly the graphql-request one [3]

- THE MAGIC: Generate your sdk by pointing graphql-codegen at your Django GraphQL schema!

- Add react-query, use the newly generated graphql-request client [4]

From here you've enabled the best of frontend technologies (NextJS, react-query) with a fully typed SDK generated by your familiar Python/Django bindings.

You can then move forward to add GraphQL based WebSockets via django-channels [5] + django-channels-graphql-ws [6] that update your existing react-query caches. Combine this with background Celery workers that make push to django-channels via Redis and you've got a a UI that auto updates based on background tasks as well.

[0] https://www.djangoproject.com

[1] https://docs.graphene-python.org/projects/django/en/latest/

[2] http://nextjs.org/

[3] https://www.graphql-code-generator.com

[4] https://react-query.tanstack.com

[5] https://github.com/django/channels

[6] https://github.com/datadvance/DjangoChannelsGraphqlWs

[Edit] Line formatting

And that's why I love so much NestJS and its code first approach.

https://docs.nestjs.com/graphql/quick-start#code-first

Same opinion. Graphql With nest.js (code first) and prisma is nice!

Similar benefits without codegen (based on decorator magic) for a python based stack:

https://github.com/adsharma/fquery

  * Use dataclasses for both database schema and the user facing operations
  * use decorators to generate django models
  * use decorators to generate graphql operations (uses strawberry-graphql)
  * Supports a method chaining query language, that can map to SQL

No code generation. Declarative mapping between database schema and web API schema possible, but not implemented.

You can get a lot of types on the server without the need to write any types manually or having to generate them by using https://github.com/sikanhe/gqtx

side note, the *.wundergraph.dev ssl certificate seems to be expired. I was testing out your service.

Hey, thanks for bringing this up! It's fixed now.

Reply on news.ycombinator.com