Hacker News Reader

Comment by atombender | Hacker News Reader

Though I dislike the hyperbolic tone and personal attacks, the author isn't entirely wrong. There are many design choices in Protocol Buffers that seem directly related to the scale and complexity at which Google operates, and which sacrifice safety, clarity and language integration.

The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code generator, but the Go generator is just egregiously bad. There's no way to write a client or server that uses the Protobuf structs directly as first-class data types without ending up looking like a ball of spaghetti, at least not if you venture into the realm of oneofs, timestamps or the "well known types" set of types.

I think we're at a juncture where we really desperately need a better, unified way to express schemas and pass type-safe data over the network that flows through these schemas. I'm currently working on a project that involves both gRPC (which uses Protobuf), GraphQL and JSON Schema, with the backends written in Go, and the frontend in TypeScript, and the overlap between all of these is ridiculous. TypeScript is a fresh breath of air that mostly solves the brittleness of JavaScript's type system, and it's absolutely crucial to extend this type-safety all the way to the backend.

The challenge is that no language is the same, and a common schema format ends up targeting a lowest-common denominator. For example, GraphQL errs (in my opinion, wrongly) on the side of simplicity and doesn't support maps, so now you have a whole layer that needs to either custom types (JSON as an escape hatch, basically) or emulate maps using arrays of pairs, neither of which is ideal.

View on HN

An approach I've used on the Go side is to build nice native Go types and converters to/from proto representations for wire marshaling. There's a minor performance hit but it's insignificant for what I'm doing. The result is quite nice, and you can largely ignore protobufs except for defining service APIs (which it's actually decent at).

Which tool are you using for TypeScript protobuf code generation? This is the next step for me, and I'd be keen to hear more of your experience there.

Yes, we use the same technique for our larger apps. We treat the Protobuf layer as a distinct layer related to the API; we have two packages, "protofy" and "unprotofy", so in the server implementation, you do something like:

  func (srv *Server) GetFoo(
    ctx context.Context,
    req *proto.GetFooRequest) (*proto.GetFooResponse, error) {
    foo, err := srv.db.getFoo(foo.ID)
    if err != nil { ... }
    pFoo, err := protofy.Foo(foo)
    if err != nil { ... }
    return &proto.GetFooResponse{
      Foo: pFoo,
    }, nil
  }

Obviously, a bit less pretty in reality. But the principle is the same. Having two packages makes it easier to read — e.g. protofy.Foo() is always about taking a "native" Foo and turning into a *proto.Foo, and unprotofy.Foo() is the reverse.

For TypeScript, I'm using ts-protoc-gen [1]. The weird part, which I don't fully understand, is that the serialization code is emitted as JavaScript code. All the type definitions end up in a .d.ts file, but the client is a .js file. Which means you still get type safety and autocompletion, just like plain TS, but it's still weird to me.

[1] https://github.com/improbable-eng/ts-protoc-gen

I believe that means you're more or less using it as intended. Protobuffers are intended for serialization. In go, serialization is often handled at a separate layer from the business logic.

You might consider https://godoc.org/github.com/tmc/grpcutil/protoc-gen-tstypes

That’s what you should do with all serialization, but everyone always thinks it’s a good idea to tie the model object to the serializer object at the beginning of a project, and by the time you are ready to build a time machine to kill the person who made that mistake, it’s much too late to do anything but rewrite the whole thing.

Could you elaborate on what you don't like about the generated Go code?

My main complaint initially was that everything was a pointer, which made constructing things harder.

Then I realized it made zero values much easier to implement. And it lets you foo.GetA().GetB().GetC() without all the intermediate steps, which feels like a violation of the Law of Demeter, but in a glue language I think it's probably the right trade-off.

Yes, you can't use them as classes/types themselves, but you're not supposed to subclass them in any language, so they're really better thought of as structs/dicts.

Oneofs, at least with gogoproto, are quite unpleasant to work with. There's a separate intermediate type generated for each member of a oneof field. With the following message:

    message Foo {
        oneof bar {
            Baz baz = 1;
            Qux qux = 2;
        }
    }

You'll have the types Foo, Baz and Qux like you'd expect, but you also get Foo_Baz, which does nothing but wrap the Baz for use with the Foo.bar property, and Foo_Qux, which does the same thing but for Qux instead of Baz. To make matters worse, the types all implement an interface which is declared but not exported. Stringing the whole thing together to assign to a oneof looks like this:

    foo := &rpc.Foo{Bar: &rpc.Foo_Baz{Baz: &rpc.Baz{}}}

That example makes it look better than it really is; in practice, you'll probably have more properties and longer names.

I'd like to humbly suggest that we use JSON please, in particular:

JSON + JSONSchema[0] +/- JSON Hyperschema[1] +/- JSON LD[2]

It's a bit to learn but I promise you, it's worth it. The technologies are not redundant (jsonschema spec is for validation, hyperschema spec is for specifying how you interact, and LD is for semantics like language and more). If you take a few hours, read all 3 specs, you're almost guaranteed to imagine a new (albeit somewhat daunting) future, where APIs and clients can be smarter.

There's no reason I can think of that a sufficiently specified JSON schema cannot be automatically converted into a efficient binary format. I have absolutely zero reason to use Protobuf and it's whole ecosystem of tools if that's the case, I'd much rather go with tools built on a standard like JSON.

GraphQL's functionality is a subset of the above set of tools -- IIRC GraphQL can basically be boiled down to an option (or two) on a resource endpoint -- How you query on the server side does indeed change a bit to support the syntax, but this is basically the same as if you'd implemented with horizontal filtering like in POSTgrest[3], or resource embedding feature[4] (which everyone ends up doing at some point with REST-ish APIs). I honestly don't know who or what is trying so hard to drive GraphQL adoption so hard but I think it's mostly hype, and in terms of general technology, by default it forces clients to know the inner workings of the data model and that's bad for any kind of future in which you don't want to have to learn X APIs to interact with X systems.

BTW I absolutely love a good well-supported rant that agrees with all my biases (yes, Google as well as every other company has a bunch of amateurs, as well as a small amount of very skilled engineers), and all the problems noted with protobuf are real problems, and this viewpoint is sorely missed in all the talks and conferences where people are touting GRPC as the next sliced bread with Protobufs as the transport (you can change GRPC's transport layer to be JSON for example[5])

[0]: https://json-schema.org/

[1]: https://datatracker.ietf.org/doc/draft-handrews-json-schema-... (also on json-schema.org)

[2]: https://json-ld.org/

[3]: https://postgrest.org/en/v5.1/api.html#horizontal-filtering-...

[4]: https://postgrest.org/en/v5.1/api.html#resource-embedding

[5]: https://grpc.io/blog/grpc-with-json

I'd love to see something with the approximate structure of JSON, but less JavaScript-bound syntax. EDN has caught my eye, but then again I've always been fonder of Lisp-y syntax as a whole.

ooh I'd love EDN as well -- but I don't think people are willing to adopt S-expressions :(

Yeah EDN is a great format. It should be a lot more popular than it is.

Proper numeric types would like a word with you. (In particular, just look into how you would get infinity/NaN in there. Fun times.)

I mean, yes, you can do everything by just passing the string representation. Not exactly efficient, though. And most schema attempts in json are usually less than compelling.

You're absolutely right that it is a sticking point, but custom types can help this. Also, I don't know about you but I don't very often have Infinity or NaN as inputs that I want to see in my schemas...

Also, the JSON schema is an evolving document being developed in the open, it can be improved over time (they're already on Draft 7) -- if numeric types are lacking then suggest a way to make them better.

> And most schema attempts in json are usually less than compelling.

I'm not sure 100% what this means -- jsonschema is pretty easy to read and works pretty well for validation, people are already being very productive with it, what is not compelling?

I've wanted NaN and Infinity on a few serialization worlds. Mainly because I didn't want to have to reinvent what the IEEE numbers already have.

My complaint on the items being less than compelling is that they almost instantly devolve into bitter fights about what I'm supposed to care about. I fully grant that XML was too heavy in much of its schema attempts. However, it is much easier to reason about the extremity of XML than it is where in the middle road I want to be in.

That it is an evolving document being developed in the open is worthless if it is not always maintaining backwards compatibility. And, since there is no way to maintain backwards compatibility now that that ship has sailed, I'm skeptical of it.

I'm sure it will hit a good maxima at some point. But "is pretty easy to read and works pretty well for validation" was also fairly accurate for most XSD work. "People are already being very productive with it" was absolutely accurate. It was only when people started to stretch with it, that things got obnoxious. And we fell back on JSON not because it was somehow technically superior, but because it was technically easier.

> My complaint on the items being less than compelling is that they almost instantly devolve into bitter fights about what I'm supposed to care about. I fully grant that XML was too heavy in much of its schema attempts. However, it is much easier to reason about the extremity of XML than it is where in the middle road I want to be in.

Could you give a more concrete example? I'm really not following -- JSON + jsonschema is almost equivalent to XML + XSD in my mind, what was it that XML/XSD gave you that didn't devolve into bitter fights?

> That it is an evolving document being developed in the open is worthless if it is not always maintaining backwards compatibility. And, since there is no way to maintain backwards compatibility now that that ship has sailed, I'm skeptical of it.

This is a pretty extreme statement, if it's not hyperbole I don't think we could ever agree. I'd much rather live in a world where bad decisions can be overturned/corrected in a sufficiently new version when a big enough problem is found. Backwards compatibility is important but I don't subscribe to the thought that any non-backwards compatible changes make a spec or a document worthless.

> I'm sure it will hit a good maxima at some point. But "is pretty easy to read and works pretty well for validation" was also fairly accurate for most XSD work. "People are already being very productive with it" was absolutely accurate. It was only when people started to stretch with it, that things got obnoxious. And we fell back on JSON not because it was somehow technically superior, but because it was technically easier.

Yeahhh I don't know if "pretty easy to read and works pretty well for validation" is true from XSD and the other XML ecosystem tools. Complexity was everywhere, and it introduced many bugs and critical security issues.

Also, I think that JSON is not only technically easier, it is technically simpler (in the Rich Hicky Simple vs Easy sense). JSON does not have potential cleverness built in, and that means it will likely never be what XML was (or at least all the complexity built on top will be very very opt-in).

XML/XSD did devolve into many fights. I will not argue against that. The main outcome was the wide adoption of JSON.

What XML/XSD did give was ridiculously easy to implement autocomplete in any xml editor that was decently featured. And it gave people a ton of rope to try and over specify things.

Jsonschema seems poised to repeat the rope mistake. That is, I'm of the mind that the anti-fragile practices of schemaless JSON have actually been a boon to its success. Trying to solidify things back to a schema gives me little reason to think it will be better this time around.

I definitely agree it would be good to have bad decisions over turned. However, I will also not adopt a technology that is a constant grind to retread the same ground over and over. At least, not if I can avoid it. This puts me in the position of not trusting the likes of committees that seem to think throwing out decisions of a scant few years ago are a good idea.

The cynic in me thinks places do it so that they can just stay faster than everyone they are assigning migration tasks to.

My assertion for "pretty easy to read and works pretty well for validation" was not broadly applied to everyone. In particular, it failed at cross vendor efforts. However, the original claim was merely the vague "people", and I stand by that being just as true for XML as it was for JSON. With similar levels of complexity regarding different bugs in parsers and whatnot.

> Jsonschema seems poised to repeat the rope mistake. That is, I'm of the mind that the anti-fragile practices of schemaless JSON have actually been a boon to its success. Trying to solidify things back to a schema gives me little reason to think it will be better this time around.

Yeah I think I agree -- while considering your point I found myself wondering what made jsonschema any different this time around...

> I definitely agree it would be good to have bad decisions over turned. However, I will also not adopt a technology that is a constant grind to retread the same ground over and over. At least, not if I can avoid it. This puts me in the position of not trusting the likes of committees that seem to think throwing out decisions of a scant few years ago are a good idea.

Not like you need me to verify but that's an absolutely reasonable stance, the timeline of a standard and associated versions is important in deciding whether to use it or not.

> My assertion for "pretty easy to read and works pretty well for validation" was not broadly applied to everyone. In particular, it failed at cross vendor efforts. However, the original claim was merely the vague "people", and I stand by that being just as true for XML as it was for JSON. With similar levels of complexity regarding different bugs in parsers and whatnot.

I don't think I agree, but we can definitely agree to disagree here. XML was certainly easy to read (if you didn't use any advanced features) and as good for validation, but I think the simplest feature-complete JSON parser is more complex than the simplest feature-complete XML parser one could write -- but then again, maybe all the features I'm thinking of that XML had weren't in the core spec (I haven't read it).

To be fair, I fully acknowledge I'm being cynical. And, to that, I'm not proud.

No worrieds I'm pretty cynical myself which is why I'm pretty surprised when I see technology that actually makes me believe things could be less-shit in the future (hyperschema + json LD gave me that feeling).

I don't think GraphQL is over-hyped at all. Maybe it's flawed, but the design is absolutely on the right traack. GraphQL completely changes how you work with APIs in a front end.

I work on React apps, and by using GraphQL, a component's data requirements can now be entirely declarative. For example, a component can do this (simplified):

  <Query query=`{
    posts(limit: 10) {
      title, creator { name }
    }
  }`>
    {({data, loading, error}) => {
      return <ul>
        {data.posts.map((title, {creator}) => <li>
          {title} by {creator.name}
        </li>)}
      </ul>
    }}
  </Query>

The component knows what it needs to render, so it declares that. With TypeScript, you can get type-safety all the way from the backend to the frontend, which means your IDE (e.g. VS Code) can correctly autocomplete, say, "creator." and suggest "name". It's rather magical.

I work on a product called Sanity [1], which evolves GraphQL one step further. It's a Firebase-like data store with schemas and joins, and a structured data model. Without implementing anything on the server end, you can run queries like these:

  *[_type == "post" && published] {
    title,
    creator -> { name },
    topPosts: *[creator.id == ^.creator._ref] {
      _id, title
    } | order(viewCount desc)[0..10],
    photos: photo -> {url}
  } | order(_createdAt desc)[0..100]

This doesn't just follow a child object (creator), it also joins with an unrelated set of objects (topPosts, which finds the top 10 most viewed posts created by the same user) as well as joining a bunch of 1:1 relations. I'm totally biased, but I think it's a game-changer when it comes to writing web apps, because you can just dump the whole query in a React component, no state management or API writing required.

[1] https://www.sanity.io/

Nothing about what you just posted couldn't be done with a normal RESTful endpoint with sufficient support for column-level filtering and embedded item filtering.

Your post is the perfect example of GraphQL is being over hyped. I absolutely get that there's a benefit to filtering at both these levels, and that the DSL cuts down on noise and gives you a way to "query" without thinking of web requests, but it's not a leap in thinking -- teams that have to deal with mobile environments have long been strapping on filtering options to endpoints to avoid sending unneeded bytes to mobile devices.

Yes -- GraphQL standardizes this stuff, but it does it in a way that is basically not compatible with anything else.

> The component knows what it needs to render, so it declares that.

??? The component doesn't know anything, components don't think. You're describing it as if you just gave the component a list of entities but you actually wrote a query DSL -- that's how the component "knows" -- you told it.

What GraphQL is doing for you here is:

- Enforcing consistent access patterns (which is how "posts" gets translated into the right URL)

- Ensuring "limit" is supported on the endpoint

- Ensuring horizontal filtering is supported

- Ensuring embedded entities get returned and they're filtered

This is not a paradigm shift. It's better, maybe, but that DSL will absolutely fail you at some point, when you try do a more dynamic query, and you'll have to drop back to writing code that looks a lot like life did before GraphQL.

> With TypeScript, you can get type-safety all the way from the backend to the frontend, which means your IDE (e.g. VS Code) can correctly autocomplete, say, "creator." and suggest "name". It's rather magical.

This is basically orthogonal... Write well typed javascript and your IDE is going to be able to help you out.

> Firebase-like data store with schemas and joins

You've lost me here. The excerpt you've posted looks even worse than SQL. At that point why not just send SQL directly to the backend (as long as you can get the permissions right and your DB is secure enough)?

Could one invent an ad-hoc REST API to perform joins, apply parameters and so forth? Of course. That's what people have been doing. That means every solution is ad-hoc. People get tired of this.

Have you tried any of the GraphQL client tools? You can literally point a client at an arbitrary server and not only see its schema, but also interact with the API. REST doesn't give you that, because the authors of REST neglected to actually specify anything beyond some fuzzy principles.

> ??? The component doesn't know anything, components don't think.

This is being obtuse. You know perfectly well what I meant: That the component localizes the information needed to make an informed request. This is domain knowledge encoded in the component.

This is contrary to how most developers design clients, where they might call an endpoint such as /posts?limit=10, which happens to fetch all attributes, even if the component doesn't actually need all the attributes.

> You've lost me here. The excerpt you've posted looks even worse than SQL.

Before a summary (and, honestly, tone deaf) dismissal such as this, I recommend reading up on it a bit [1], and maybe trying it out. SQL is great, but it does not handle nested documents/relationships, so you end up with a lot of messy structural mapping between flat relational data and structured data, which is why projects like Rails/ActiveRecord and Hibernate are so popular.

For example, a query such as this:

  *{
     name,
     photos->{ url },
     friends->{ name }
  }

will return something like:

  [
    {
      "name": "Bob",
      "photos": [
        {"url": "http://..."},
        {"url": "http://..."}
      ],
      "friends": [
        {"name": "Jane"}
      ]
    },
    ...
  ]

The point isn't that you cannot, with a sufficiently complex ORM and enough elbow grease, do that with SQL. It's that it should be unnecessary, because developers shouldn't need to write an entire data layer every time they want to bring an app up. This is part of what GraphQL brings to the table, too.

[1] https://www.sanity.io/docs/data-store/how-queries-work

GraphQL certainly has a cleaner/less involved interface, but it's also less interoperable. My problem with it is that it doesn't offer much value for the amount of effort, and it isn't the paradigm shift it's claiming to be in the first place, it's just being marketed well. HateOAS[0]+Swagger[1]/jsonschema hyperschema[2] is enough to do what GraphQL does.

> This is being obtuse. You know perfectly well what I meant: That the component localizes the information needed to make an informed request. This is domain knowledge encoded in the component.

> This is contrary to how most developers design clients, where they might call an endpoint such as /posts?limit=10, which happens to fetch all attributes, even if the component doesn't actually need all the attributes.

I apologize, that was a terrible way to phrase my objection.

But I want to note that basically GraphQL has offered you a way to avoid writing:

    axios.get("/posts?limit=10&filter[0]=prop&filter[1]=otherProp&fetchEmbeddedEntities[0]=thing")

My point was that you can tack on a not-string query API that declaratively builds this without committing to "implementing a GraphQL endpoint" for all your APIs -- the technology is already there, what people are lacking is shared structure. GraphQL does create shared structure, but in an all-or-nothing way and I avoid building with tools that do that.

BTW, as far as actual dynamic API recognition goes, the promise of jsonschema + json hyperschema + json LD is much more promising (it's basically equivalent to the semantic web promise) -- GraphQL is a step in the right direction but the lack of interop means we have to step backwards to go in any other direction.

Let me put it this way, could you imagine writing a query where you don't know the name of the model on the server side? Like you only know the thing you want (let's say a "vehicle"), and you know the backend has "vehicles" but you don't know what they're called? Being able to do that query is a paradigm shift, and it's possible with the tools I mentioned, though the promise is yet to be realized by and large.

> Before a summary (and, honestly, tone deaf) dismissal such as this, I recommend reading up on it a bit [1], and maybe trying it out. SQL is great, but it does not handle nested documents/relationships, so you end up with a lot of messy structural mapping between flat relational data and structured data, which is why projects like Rails/ActiveRecord and Hibernate are so popular.

I've tried it -- I'm not impressed, this is the crux of my point and I haven't seen anything yet that can help me change my mind. It is a fact that the chunk of code you posted as a query is a nightmare to look at. Yes, my brain will eventually

Despite how people try there aren't many query languages that can match the expressive power of SQL. For all it's warts, it is excellent at what it does.

> SQL is great, but it does not handle nested documents/relationships, so you end up with a lot of messy structural mapping between flat relational data and structured data, which is why projects like Rails/ActiveRecord and Hibernate are so popular.

This is a wildly inaccurate statement. SQL is for querying relational database systems. Well structured relational data is the most structured data you're ever going to find. ORMs (ActiveRecord/Hibernate) are:

- excellent at reducing boilerplate when it's a perfect fit (pro) - often used by people who don't understand the expressive SQL (con) - great at creating N+1 query problems (con) - great at stopping you from using the deeper features of your DB (con)

ORMs are easier, that's why they saw widespread use.

It's been my experience that people fall out of love with ORMs as they reach their rough edges (which they must have, they are leaky abstractions). I much prefer query builders/generators and in-language level abstractions (methods/functions/etc).

> For example, a query such as this:

Please stop posting queries, I get the query language, the DSL is fairly easy to understand (this is great for GraphQL).

> The point isn't that you cannot, with a sufficiently complex ORM and enough elbow grease, do that with SQL. It's that it should be unnecessary, because developers shouldn't need to write an entire data layer every time they want to bring an app up. This is part of what GraphQL brings to the table, too.

I think you've conflated an ORM with a sufficiently capable and metadata-tagged backend. These features aren't ORM level feature necessarily, look at Postgrest's vertical column filtering[3] and embedded entities[4].

Also, my original point was that GraphQL is over-hyped considering that it is not a paradigm shift, and requires you to do a bunch of non-interoperable work aside from REST without much benefit.

I'm going to find some time and implement GraphQL automatically (with the help of jsonschema and hyperschema, etc) on top of a regular RESTful endpoint, I'll post it to HN when I do.

[0]: https://en.wikipedia.org/wiki/HATEOAS

[1]: https://swagger.io/specification/

[2]: https://datatracker.ietf.org/doc/draft-handrews-json-schema-...

[3]: https://postgrest.org/en/v5.1/api.html#vertical-filtering-co...

[4]: https://postgrest.org/en/v5.1/api.html#resource-embedding

> GraphQL has offered you a way to avoid writing:

> axios.get("/posts?limit=10&filter[0]=prop&filter[1]=otherProp&fetchEmbeddedEntities[0]=thing")

Is that right? I've never used GraphQL, but I thought the main advantage was not that the client didn't have to write that, but that the server didn't have to write the handling code for the joins to the embedded entity (to pick one example), or any of the dozens of other possible joins someone might want to do for any given relation.

GraphQL is more or less a data description language. It describes queries and mutations. There are JS libraries that take a GraphQL query/mutation, convert it into the spec'd JSON payload, send it to the backend (single endpoint), and map the requested fields back into a JS object to be used. This is super nice when paired with React -- it's hard to overstate how nice it is.

On the backend, there are libraries that process that JSON payload into whatever your framework wants (dicts in Python, for example). From that decoded payload, you have to figure out what to do. If you're using Django and you get something like

    {'name': 'Ruth', 'age': 85}

to your 'People' query, you need to ask your ORM for People named Ruth who are 85 years old, and return them. It is essentially exactly like REST API design, with a few important differences:

- You don't have to worry about irrelevant HTTP stuff. What status code is appropriate? What HTTP action? Isn't this URL a little wrong? All irrelevant.

- There is no standard for pagination or ordering. In fairness, REST's answer (do it in query params... just like everything else!) is a little unsatisfying. But REST never made grandiose claims about data description. You go through the whole GraphQL site and it's more or less just an exposition in how great it is to just _describe_ the fields you want to retrieve/change, but nothing about "what if I just want the first 10,000 results?" This is, as you might imagine, an important and common issue.

- There is no (meaningful) standard for errors.

I think it's a step in the right direction from REST. URL-based APIs really just never earn their keep, HTTP actions are too limiting, versioning weirds everything, query params naturally grow to contain/become their own DSLs. But as far as backend engineering goes, GraphQL really only addresses the problems we didn't really mind (URLs/HTTP actions) anyway. To answer your question directly: the benefit of GraphQL is... 98% realized only by API clients. The backend is still doing everything it used to.

This is an excellent response and is 100% correct.

At this point all I can say about my stance is that I would have preferred a standard for HTTP+JSON instead of how GraphQL is completely different.

I also completely forgot about HAL[0].

To be clear, I'm not against GraphQL, I understand it and I understand what it brings to the table but I just thought the alternative was better if only because it relied on a web of standards that were always aimed at doing more (semantic web).

[0]: http://stateless.co/hal_specification.html

Ha thanks.

I have fundamental misgivings about "APIs as bespoke ORMs for browser clients" -- at least how they're currently commonly implemented ([db] -> [orm] -> [controller magic] -> [request parser & deserializer / response serializer]). It just has this sort of... ad-hoc lack of coupling. Things like Datasette or PostgREST feel more "right", but I'm still a little incredulous that they're sufficient for even the majority of needs.

Designing a system that can handle concurrent reads/writes at production scale is hard even when you control the entire stack. When you consider all the constraints that modern web dev labors under, it's a miracle any of it works at all. But to be honest, my suspicion is that we get it to work by trading 90% of our performance potential for all this extra machinery between the client and the DB. We needed to do this 20 years ago because DBs didn't have the functionality (ex: row-level security) they do now, but at this point, maybe it's time to revisit. Either way, I'm doubtful that choosing a schema and protocol for client/server communication would meaningfully move the needle. I think a more fundamental rethink is necessary. Maybe after 3-4 years of wasm.

in relation to your concerns about PostgREST check out these links http://docs.subzero.cloud/postgrest-starter-kit/ http://docs.subzero.cloud/architecture-and-project-structure...

I've long thought about creating a standard(iana type) for PostgREST json schema + http querystring conventions. I've seen some APIs[1](only one I can remember now) that follow PostgREST conventions, so perhaps this could benefit all of us who don't want to jump on the GraphQL bandwagon but still want an expressive and interoperable/standardized way to query resources.

We currently use OpenAPI but we found some shortcomings[2] and would like to have a more tailored standard.

Right now what we're missing is more funding, we don't have a backing company like GraphQL/gRPC, we rely on community donations through Patreon[3]. Definitely consider supporting us if you want to improve the state of REST APIs, allow us to keep fighting for web standards :).

[1]:https://developers.blockapps.net/advanced/concepts/#cirrus

[2]:https://github.com/PostgREST/postgrest/issues/790#issuecomme...

[3]:https://www.patreon.com/postgrest

Thanks to you and the other maintainers for your tireless work on Postgrest (also as a member of the Haskell community)!

Is there a paypal link for one-time-donations? I don't use patreon and am not really down to increase my online footprint but would love to be able to donate (I also couldn't find any mention of taking donations on the github or in the docs @ postgrest.org... you might get more donations if it were at least mentioned?)

Thanks a lot for your feedback @hardwaresofton, I've just added a section in our README that includes a Paypal link for one-time donations.

https://github.com/PostgREST/postgrest#supporting-developmen...

Thank you for your support!

You can do this with PostgReST as well. Something like:

  GET /user?select=name,photo(url),friend(name)

I do not use PostgReST, but created something similar.

When I perform the following 3 individual requests after I performed the above request the results can all be retrieved from cache (at the server and/or client side):

  GET /user?select=name
  GET /user/{name}/photo?select=url
  GET /user/{name}/friend?select=name

Even when I do the following (without the photo(url) part) it can now just build it from the cache without hitting the database server:

GET /user?select=name,friend(name)

That is one of the advantages of using ReST. It allows you to utilize and benefit from already existing proven and well defined infrastructure components (the connectors etc.).

GraphQL can be used on top of this as well. I experimented with this once as an alternative to the PostgReST-like syntax.

Someone linked to this thing called subZero[0], which seems to be a starter kit that exposes APIs as GraphQL and REST from a Postgres database (it uses postgrest), maybe it's a good example of them working side by side.

[0]: http://docs.subzero.cloud/

The other problem with GraphQL is letting external clients make arbitrary queries is a denial of service attack in waiting, and even with internal clients, how much can you trust another department? Having ugly, non proper REST endpoints adds a negligible amount of implementation time to the client but lets the server ensure it can actually meet its performance goals.

I have seen research in this problem area, where one is capable of knowing ahead if the query is legit or not. Anyway for internal purposes it's fine

Small correction -- I meant vertical filtering in postgrest[0]

[0]: https://postgrest.org/en/v5.1/api.html#vertical-filtering-co...

Yup. No stream API to unmarshall protobuffers suck majorly. You have to re-invent framing

Which language are you talking about? C++ proto is built around CodedInputStream which is what it sounds like. You can use it with or without the generated message code.

https://developers.google.com/protocol-buffers/docs/referenc...

Golang

Reply on news.ycombinator.com