Hacker News Reader

The point isn't to write less code -- especially server-side code.

The point is to provide a consistent, feature-rich, declarative, verifiable and performant API that is easy for the client to develop against.

SQL is an excellent analog -- no, it doesn't make the SQL server implementation easy, but it's a consistent, declarative, and powerful API. "Even" business analysts can write SQL queries.

If you don't have a strong incentive to make life easy for the client, the case for GraphQL isn't as strong.

Great point about not focusing on code reduction. GraphQL is such a big change from rest that the primary benefit will be different for each team you ask. Form my perspective in a startup with ~50 developers is this:

By providing a clear and powerfull interface you can significantly reduce inter-team communication leading to both higher throughput and faster iteration cycles.

From a technical perspective reducing round trips is important, especially on mobile devices and GraphQL makes this trivially easy.

These two points I suspect is the primary reason many medium+ sized development teams end up implementing their own bastardised version of GraphQL in their existing rest api. Many people we talk to at http://graph.cool are interested in deploying a thin GraphQL wrapper on top of their existing api for exactly this reason.

GraphQL works really well for some edge cases, and those edge cases will be more evident if you're FaceBook. For smaller projects, the benefits of integrating GraphQL libraries on the client and the server v/s special handling for a limited set of APIs is not obvious.

By special handling, I meant:

    https://example.com/friends?fields=name,pic,posts&posts_len=10

The vast majority of APIs will not require this kind of result-shaping. And for those that do, passing comma separated fields (using a protocol teams already understand) is worth trying before adopting something more drastic. Of course, YMMV.

Often what gets lost in these kinds of discussions is the difference between what is theoretical and what is practical. Yes, you can have a perfectly designed REST API which has all of these features.

What happens, though, when you want a new piece of data? Do you redesign the API to fit this new piece of information into it or do you make a new endpoint? What if the new piece of data doesnt actually belong with `friends` but if you do the fetch for friends and this new piece of data at the same time you can get a performance boost? Do you make a new endpoint `friendsandfamily`?

Now, what if you want to migrate all calls over to that endpoint, you now need to figure out if any of your frontend code or legacy systems are still using that old endpoint before you remove it.

The maintenance of that system is much higher because you've conflated what data you want with how you want to get it.

how is this solved by GraphQL though?

You still need to add a new edge `friendsandfamily` (same as adding to a new endpoint) or add a new a field to the existing object (same as adding a new field to the existing endpoint).

What am I missing?

(1) GraphQL returns only what the client asks for. No need to worry that adding to an existing resource will increase response times or bloat the response for existing queries.

(2) If you decide to go the friendsandfamily route, you'll just be adding `family`. You won't have `friends` endpoint and `family` endpoint and `friendsandfamily` endpoint. A GraphQL schema doesn't increase combinatorially in size; rather the possible compositions increase combinatorially.

for 2) isn't this the same as adding an optional "include=family" to the existing friends endpoint?

This is what the JSONAPI spec suggest too.

Yep. You can use query parameters to fix 1) and 2).

Go down the rabbit hole a little bit further, and you have GraphQL (if you're lucky....if not, you'll just have a mess).

I gave a talk about this recently [1]. One of the subtle points of why GraphQL has the potential to be more productive for smaller teams & projects is because it is well specified. Standards are a bit constraining, but they're also enabling. There are already some nice tools built on top of GraphQL and more will come as the project matures. This is of course the same value proposition as Swagger, JSON-API, and other formalizations of REST. In my opinion those projects are likely more mature today - especially for native mobile apps. But in a few years? I suspect GraphQL will be more productive even for small teams.

[1] https://www.youtube.com/watch?v=P0uJHI7GjIc

Even when it does require result shaping you can just use a non-standard-REST POST request where the requested information is in the HTTP body. Implementing GraphQL on the server side seems like a lot of effort for no immediate gains. I am pretty sure it works very well for Netflix and Facebook though.

Does GraphQL actually provide performance advantages right now? From what I've read about it, it seems like it suffers from a bit of an N+1 queries problem if you don't do a lot of custom optimization, which seemed like considerably more work than a REST API.

Just like SQL, GraphQL needs a good query planner for complicated queries.

REST is more like a NoSQL database where you write all the joins by hand: on the plus side, you get to control the query execution, on the negative side, you have to control the query execution (also you have round-trips and lack some possible internal optimizations).

The "performance" of GraphQL refers to the fact that most things can be queried in one round-trip, rather than many for RPC, REST, or vanilla HTTP. Each of these are abstractions, not implementations.

So the maximum performance of GraphQL is better than the maximum performance of REST, particularly over mobile networks...as you said though, the devil is in the implementation.

fwiw I think Facebook's current implementation interfaces with a read-through cache. I think they've managed to make it fast without having to spend all of the effort on query planning that way.

I also built a prototype with it. IMHO the best feature is strongly typed data and introspection, which enables lots of tools (like GraphiQL). But you can build something like this for traditional RPC APIs too - that's not necessarily related to the other GraphQL features.

The next advantage is of course letting the client define what he wants to retrieve in a single query. Whether this is useful depends on your application I think. For mine - while it was cool to use - it had no big benefit compared to having a set of orthogonal RPC APIs that the client could use to fetch the information it needs. Less round trips are a reason, but if you have your API on HTTP/2 or something else which supports small queries efficiently it might not have a too large impact.

The other large difference is that with a "lots of small APIs" approach you have to compose the data on the client while with GraphQL it's done on the server. One advantage of doing it on the server makes the load of it more predictable, since the complexity of the queries is well known and there can't be clients which try to issue queries with an abnormal amount of complexity.

What I disliked most about GraphQL were mutations. The ability to include multiple mutations into a request made it sometimes hard to reason about which of those succeed or errored based on the query result. I also don't see a huge benefit of specifying a complex query result for a mutation - as I am also now one of the believers that queries should be separated from mutations/commands and that the mutation mostly needs to report back whether an operation succeeded.

So all in all I now think it is an interesting operation for a Query API, if the target application requires complex user-defined queries. But I would rather not use it if clients mostly query the same amount of data or for command/control APIs.

You can compose data on the server without GraphQL. PostgREST does this (ironically not very "RESTful") thing: http://postgrest.com/api/reading/#embedding-foreign-entities

Would it be possible to write the server part such that either all mutations succeed, or none of them do?

I've written an interface similar to GraphQL some time before it was made public using JSON to describe the desired data tree, filters, etc.

The real upside of GraphQL is that it has become popular enough that there are parsers in most languages, and that you don't waste time designing and testing a new standard. If you use it for a public API, you can also simply point people to the existing documentation.

It's definitely not a widely accepted standard, but still better than rolling your own!

A question from someone that has never used GraphQL: what makes it better than SQL? You don't even need parsers in different languages for SQL, you could just send it straight to the DB. But I assume I'm missing something.

You might be confused by how PostGraphQL works, as it makes the queries very close to the database structure.

In real life there will usually be a lot more going on to serve a GraphQL query than simply translating it to SQL.

The data you'll send might come from many different sources, including any kind of database, caches, or just constants in your code.

As commented here, permissions over data is a big topic, and if you were accepting raw SQL, you'd have to parse / tokenize it, and somehow validate it before executing it. This is what GraphQL does, but instead of trimming down and restricting the usage of SQL, it defines a language that is similar, close to being a subset of it, with a much simpler syntax.

You can allow predefined SQL strings and define security rules for each string. If you need new query, you have to write new security rule for it. Won't work for dynamically generated SQL, but for most CRUD SQL it should work.

If you have N predefined SQL queries, then why not just wrap them in REST endpoints (it sounded to me that untog was suggesting that SQL be the interface to the data). If the user can write whatever query they want, then validating by string isn't really a good idea...

> You don't even need parsers in different languages for SQL, you could just send it straight to the DB.

You probably don't want to do that for security reasons. So either way you need a parser, and at that point, it makes sense to use something that is database agnostic and allows you to model relationships without complex joins or otherwise exposing the internal representation of the data.

Clients can use it to request exactly what they need. Since it's a graph, you can just keep grabbing nodes to get the fields you need in the shape that works for your client. GitHub added a GraphQL endpoint for their API, the improvements they cited were reduced overfetching of data the client didn't use e.g. urls to linked resources, and fewer underfetches e.g. using the results of the first request to populate a second or third request.

http://githubengineering.com/the-github-graphql-api/

It's not really GraphQL vs SQL, it's GraphQL vs RESTful APIs.

I have done this same, rolling my own. We copied a popular public API, LinkedIn I believe. We called it "api selectors".

> You'd end up writing about the same amount of code that you'd have written with a standard REST-style interface.

With every fad that has hit backend development, ORM/move-everything-to-frontend/merge frontend and backend a la Meteor/GraphQL/etc, there have been promises of simplicity and less code.

The problem, as you discovered, is that the overwhelming majority of non-CRUD code on the backend is directly related to security and cannot be abstracted away or moved anywhere else. You end up dancing the same dance, no matter what paradigm you use and some of them make that dance a bit more awkward.

Personally, I've settled on Swagger a few years back. There's no fancy paradigm, it's just plain REST with an open sec. Because you define your API ahead of time, you can automate CRUD, automate validation, testing, and documentation. All that's left is the nitty-gritty which you have to do no matter what and it doesn't have an opinion on how that should be done, that part is up to you.

it can be moved, to the database itself (constraints, grants)

We have an app running on graphql with 3 different user roles.

I'm not sure what other people do but on field resolvers we check for the current users access level and see if they should be seeing this thing. If not, the resolvers return errors.

In a more elegant approach, we have some of our object types duplicated / inherited. Say, "User" and "UserAdminView". If an admin queries user(id:3) they receive "UserAdminView". If a user does that, they receive a watered down "User" type, which exposes only the fields users can see. UI then selects the appropriate fragment.

Maybe there are much better ways though.

Writing the same amount of code is OK.

Doing one round-trip instead of several, no matter how deep and complicated your data tree is, may be the interesting part.

That can easily be achieved by not following REST so strictly.

Actually, it can easily be achieved by following REST strictly, especially the part where the key part of REST API design is selecting resources and designing appropriate representations for the intended use.

"REST" as consistently and independent of use case a trivial CRUD layer over base tables is...well, almost as from actual REST as the JSON-based RPC over HTTP version of "REST".

Sorry, can you provide examples of deeply nested result hierarchies that are expressed well with Rest?

I use JSON API which allows for "compound documents" instead of deeply nested results[1]. I would say it's well expressed, and it doesn't violate any of the principles of REST.

[1] http://jsonapi.org/format/#document-compound-documents

I am not sure I follow the conversation. What would a deeply nested result hierarchy be, exactly?

Are we talking about something like:

CREATE TABLE person (id INT, name TEXT, birthdate DATE);

CREATE TABLE car (id INT, brand TEXT, model TEXT, licence TEXT);

CREATE TABLE p_c (pid INT, cid INT, FOREIGN KEY pid REFERENCES person (id), FOREIGN KEY cid REFERENCES car (id));

And for a given person, we want to return their details plus the cars they own?

gopher://example.com/person/{id}

# Get person ID

1, Joe, 1970-01-01

gopher://example.com/car/{id}

# Get car ID

1, Ford, T, 1313

gopher://example.com/person/{id}/cars

# Get all cars owned by person ID

1, Joe, 1970-01-01, 1 Ford, T, 1313

,,, 2 Ford, S, 1717

gopher://example.com/person/{id}/cars/{id}

# Get specific car

gopher://example.com/car/{id}/owners

# Get owner(s) of car ID

gopher://example.com/car/{id}/owners/{id}

# Get specific owner

gopher://example.com/car/{id}/owners/{id}/birthdate

# Get specific owner's birthdate

gopher://example.com/car/{id}/owners/birthdate

# Get birthdate of all owners (assuming no domain overlap between IDs and field names, other solutions possible otherwise)

If this is the sort of thing we're talking about, I've got that t-shirt and assumed everyone had too, so I guess I'm missing the point here. By how much?

You seem to have messed up the format of your RFC4266 gopher:// urls. The first character of the path is the type, not part of the selector. Neither types 'c' nor 'p' are standardized, but I believe the convention is that they are calendar and ASCII-based page-layout documents, respectively.

I'm not sure what the correct type actually is, though. For JSON the closest RFC1436 type is probably 0, though I would be inclined to consider using (non-standard) type j instead.

</pedantic>

Indeed, it's been a long time and it wasn't common to actually use URLs back then!

Thanks for the example. I am also trying to wrap my head around it.

Is it possible to get all that information in one round trip ? I believe that is one of the benefits of graphQL.

> Is it possible to get all that information in one round trip ?

If we're talking about the same thing, yes.

One pattern that I use in my APIs is as follows:

Assume a JSON object like:

{ name: "Joe", colour: [ {red: 0, green: 128, blue: 90}, {red: 35, green: 88, blue: 199} ], hair: { length: { value: 9, uom: "cm"} }

I have on occasion provided an API like:

# Assume that {id} returns an object like the above.

GET /api/{id} # Returns the full object

POST /api/{id} # Replaces the object

PUT /api/{id} # Overwrites properties in the object

GET /api/{id}/name # Returns "Joe"

POST /api/{id}/name # Overwrites the name

PUT /api/{id}/name # Overwrites the name (also)

DELETE /api/{id}/name # Removes the "name" property

GET /api/{id}/colour/0/green # Returns "green". Other methods as above.

* /api/{id}/hair/length/value

PUT /api/{id}/hair/colour # Creates a new property

GET /api/{id}/colour/0;2 # Returns the first and third items in the array

GET /api/{id}/colour/1/red;green # Returns those two properties from the object. Note that this imposes some restrictions on property naming (no semicolons)

And this I never implemented, but I would if I had a need:

GET /api/{id}/colour/(0/red;green);(1/blue) # Returns [ { red: 0, green: 128}, {blue: 199} ]

The problem is that you are making up the syntax. GraphQL gives you this power in a standard that is machine readable so tools can be built around it.

Seems that way, with the added benefit of the client specifying the fields they need. Valuable if the underlying API changes to add a field but the clients can't use it yet.

I find that if I'm having to deeply nest result hierarchies into my REST payload I probably haven't spent a lot of time thinking about how the API I am building will be consumed. Usually designing my endpoints to align more closely with the concepts consumers of the API expect prevents the need to nest a lot of data. You may have to break your endpoints away from your data structures somewhat and reorganize your API into the logical structures that more closely represent the way interactions are performed in your software.

That's not to discount GraphQL, because I think it clearly has it's use cases, especially at Facebook size.

Verily!

GraphQL is one way to do just that.

REST has the virtues of uniformity and discoverability. I don't yet understand whether GraphQL possesses any of these too, though.

In theory yes, in practice, I never saw REST discoverability being used much IRL

GraphQL puts discoverability front and centre, by prioritizing introspection and static typing. GraphiQL (the browser-based GraphQL explorer) is pretty hard to beat.

Check out Github's GraphQL explorer for a great example of this: https://developer.github.com/early-access/graphql/explorer/

Because 90% of the "REST" developers don't actually build hypermedia apis, so they aren't really "REST" if you denote "REST" as somewhat complies with Roy Fielding's paper where he invented the acronym REST

Can you provide standardized examples of REST's uniforms toy and discoverability?

https://spring.io/understanding/HATEOAS

Its worth nothing that Roy Fielding, the guy who coined the term REST, doesn't consider an API RESTful unless it implements HATEOAS. At least, that's what he said in some talk I watched sometime ago.

He doesn't consider an API restful unless it is a hypermedia api, of which HATEOAS is an excellent example of.

Naming and formatting it differently is an easy way to get away from the REST dogma, though.

For me the main benefit of graphql is to have a common interface for mixed backends. One query seems unified, while it may request data from an SQL DB, the cache, redis, the file system, elastic search, etc.

It makes it very easy to switch backends, or reuse the same query on the client or the server. Microservices have one common standard language to query data, to whatever it wants.

ORMs tried to do that but they work well only for relational data. Plus unless you use JS everywhere, you can use it on the client and the server.

Anyone know if colleagues of GraphQL address this somehow? Like Falcor, om.next, Datomic, DataScript, Relay, Neo4j etc?

(I know some of those might not be the same, just throwing in a bunch of names vaguely clustered in my head)

SPARQL, the W3C standard real Graph QL, where most implementations accept queries direct over http. Have implementations that do role based security depending on the connection. Commercial implementations such as StarDog or Oracle Semantic Network among many others, will allow you to limit certain subsets of your data in many ways, and connect to many other datasources via different translators/mappers.

For the JSON developer though you would need to combine it with a nice JSON-LD formater which is not yet common among implementations.

> That means that you can't pass a GraphQL query directly to an underlying DB provider without verifying what is being requested. You'd end up writing about the same amount of code that you'd have written with a standard REST-style (or other) interface.

One benefit of committing to PostgreSQL as a backend: It already supplies a rich set of features for fine-grained security, and with views it's easily possible to add the filtering features.

This can reduce the need for much of the app middleware that does this traditionally. This is the thinking behind PostgREST, for example.

I've seen access control implemented both at database level (views and stored procedures) and at middleware level (Java/Hibernate). I'll greatly recommend the first of the two approaches, even if it means that developers have to deal with SQL and of the PL of the database. Thing is, they need to know this stuff anyway to be worth their pay. Hiding behind an ORM gives you this nice, familiar, and deceiving feeling of data just being objects which you can manipulate pretty much for free. I've seen many examples of code which load up the entire contents of tables, then proceed to manipulate it in clunky Java. Scales terribly, of course.

Hm, I'm curious. How do you implement database level access control for the generic case of website with hundreds of thousands users? Do you suggest, given a postgres database for example, to add the same amount of roles, and to have individual db connections initiated per each request?

Hmm.. may depend upon the granularity that you need I guess. As far as I recall, databases do implement access control at table level, but often you need better control than that.

In the system that I most recently worked on we had to implement our own access control logic as table level was too coarse. It was a datacenter management system which supported multiple users, each with a set of roles, and each user belonging to several companies. The data center inventory that would be returned by queries depended both upon your role (sysadm vs customer admin vs regular Joe) and upon your company affiliation. I guess I'm mostly writing this to illustrate how quickly this kind of thing becomes closely intertangled with the domain and with your business logic.

Will try to answer your questions to the best of my ability:

1) The amount of users is not a scalability issue but the number of requests may be, of course. I'll assume that's what you mean. My best advice would be to profile your system to see how it behaves at great load. If access control turns out to be a problem, maybe you are doing too complex joins. One path forward may be to "denormalize" your database scheme, i.e. accept that the same information is stored in several places for the sake of efficiency. Another idea is to spin up an Elastic Search box to help with the query that is your #1 bottleneck.

2) As for the question about individual database connections, well, I think I'd pool them or I would build upon some framework or app server which pools them. It's an easy win with standardized solutions.

Caveat: Bear in mind that although I seem to be posing as some kind of expert right now, I'm really not. I'm more of an all-round full stack kind of guy. So seek advice from multiple sources :)

You treat database roles as user groups (admin/employee) and based on that define table(view) level access. Then whenever a request is made, your api code switches to the appropriate role and sends to the database the specific user id (CUGs for example). Then you use views/RLS to filter access to specific rows.

It might be something already mentioned below here, on mobile so might miss it.

However, what we do : we use PG role row level access. E.g. each database user has explicitly defined the subset of data ih has access to. All you need to ensure your application business level views do not mix/confuse logical view of partial data. And! You choose to use correct database user in your back-end data stream/feed.

Have I already mentioned the PG is a great tool and the feature gap between MS Sql, Oracle and PG constantly narrows down as we speak?

_Edit_ typos & errors

> There are plenty of things that a client is not supposed to see, often based on the roles of the requesting user. That means that you can't pass a GraphQL query directly to an underlying DB provider without verifying what is being requested. You'd end up writing about the same amount of code that you'd have written with a standard REST-style interface.

Why would you need to write the same amount of code? The only code that would remain the same is the validation logic.

The amount of code needed to verify that only the proper fields and records are included for a particular response taking into account the user's permissions etc is often not so different from the code needed to retrieve just that data in the first place.

There is an issue here where as you give more sophisticated querying capabilities to the client, you need to have correspondingly more sophisticated validation of those queries on the server, which offsets some of the gain.

Perhaps another way of asking it: how is GraphQL different to sending SQL statements from the client? In both scenarios you'd be "only" handling validation logic and ignoring the rest.

Not much difference except that GraphQL does not specify how the data is processed.

You could answer the data out of a flatfile for all that the endpoint cares.

It also allows to secure access against arbitrary data requests, since on the server end, the resources only see the individual requests to the DB.

In my experience, it's much easier and less verbose to validate GraphQL after parsing than just validating any SQL you plan to pass to the backend.

My understanding is that different user roles would see different schemas depending on what they are allowed to see. This should definitely reduce the amount of code that needs to be written. I haven't actually implemented such an API though, so I may be wrong.

It's useful for unifying upstream providers. If all you're doing is using it to pretend the server-side application layer doesn't need to exist, it's going hurt.

Comments like these. This is why I read Hacker News

> I tried applying GraphQL to standard webapp/services projects

I'm sorry I don't understand.

Graphs are not a thing for running webapps/services. They are a subset of CS/maths aimed to solve specifics problems that can be expressed as graphs.

If you don't have a graph problem to solve, you don't need a graph database.

GraphQL is a bit of a misnomer, it's not actually for querying mathematical graphs. It's so named because it was originally for https://graph.facebook.com