Hacker News Reader

Comment by runeks | Hacker News Reader

I don't understand this, to be honest. What does type safety have to do with serialization formats or application protocols? I've used Servant (Haskell) to define the REST API for my backend, which gives me type safety, server stubs and generated client code for free. In my view, type safety is about what you internally use to represent your API, and has nothing at all to do with the underlying protocol(s). There's nothing about REST+JSON that prevents type safety, as far as I'm aware.

I plan to switch to protobuf3 for the serialization format, since this offers both a JSON and binary representation. Why would I want to choose gRPC+proto3 over REST+proto3?

View on HN

I think what they mean is by defining the contract first as a .proto file, and by having type-safe languages automatically read them and generate code, they are able to have a sort of cross language type safety.

If you create a method like

    double GetThing();

and then you want to change it to:

    int GetThing();

All you have to do is change it in your proto, then both the typescript in the browser and the go code in the server will adapt, and shout at compile time if the types don't match. This wasn't the case when the server was sending JSON to a web listener. You'd have to hunt down the dependency to that method and change it.

But unless you control all client code, you can't check the client code at server compile time.

You're still breaking and forcing a refactor by all your clients and there's no way to track that with type safety.

That said, this use case seems to be for a single web front end and go back end but that part is left out of the title.

Protos are fine, google likes protos, gRPC works for Google because they have that insane CI system that builds every project at once...but any schema would work and you can be generating and checking against a schema for a JSON API as well. You don't need to move past REST and JSON to get what you're asking for.

I am aware this is not a real life scenario, I'm just saying what he calls type-safe, is actually something like cross-language-static-typing.

GP was a bad example. You probably wouldn't have an RPC returning a simple type. It would instead return a message (aka a Go struct, or an object in JS) which has various fields (or even just a single field). Protobuf messages have built-in backwards compatibility as long as you don't change field numbers. That way you can incrementally change your messages without breaking older clients.

To change this without breaking existing users, including internal ones, you shouldn't redefine the method signature, you should add a new one and deprecate the old one.

> There's nothing about REST+JSON that prevents type safety, as far as I'm aware.

Well, except for the fact that JSON is pretty much untyped? I don't understand what there is to not get. If you want type safe RPC you have to use an RPC system that at least has types! What stops me sending `{ name: 12, age: "Hello"}` to your JSON RPC system?

> If you want type safe RPC you have to use an RPC system that at least has types!

If you want a type safe RPC, you have to use proper RPC system in the first place. It doesn't matter if it is gRPC, SOAP, XML-RPC, JSON-RPC, or Apache Thrift. REST doesn't even define serialization format or error reporting.

> you have to use an RPC system that at least has types! What stops me sending `{ name: 12, age: "Hello"}` to your JSON RPC system?

So you claim that JSON doesn't have types, right? And, by extension, Python and Lisp don't have types, too, because of the very same arguments? You know you're being ridiculous now, right?

JSON's types are limited (object, array, string, number, boolean, and null) and provides no standard way to define new types. It frequently devolves into ad hoc data structures with stringly typed values. I'm pretty sure that's what @IshKebab meant by JSON being "pretty much untyped".

I'm pretty sure that's not what he said. He used an argument that is not about how much one can express directly, but about dynamic typing nature of JSON.

JSON is just a serialization format. Everything you send over the wire is just a bunch of bytes, including JSON and protobuf. JSON has types, and you can use it with a typed RPC system just fine. Whether or not clients can deserialize something you send them can't be known statically, even with protobuf.

> What stops me sending `{ name: 12, age: "Hello"}` to your JSON RPC system?

Nothing. And the same applies to a typed RPC. You don't have to use the typed client. Or you could be using an old version, or it could be misconfigured, etc. You can enforce schema on the server and client side, but you'll never really know if it works statically, since you don't compile and deploy the server and client atomically.

Yes you don't have the guarantees of statically compiled/linked code but the entire point of using protobufs is that if you use the generated interfaces, you'll end up with fast binary serialization/de-serialization with type safety. That's a lot better than just using JSON. Which is very verbose.

You could of course make your own protocol and type system and use JSON for transmission. Why bother when this is done for you and is likely going to have the best adoption when it comes to interface definition languages like protobuf or Thrift.

I agree, I love protobuf and JSONSchema and Thrift, I think they're great ideas that solve important problems.

But the hype is getting carried away--they aren't a silver bullet for distributed systems, they're just a way to manage schema. Define a schema in an IDL, and Protobuf/Thrift/whatever generate schema validators and serializer/deserializers for clients and servers. But they are not type safe across systems, and that they compile doesn't imply anything about whether they will actually work at runtime.

Sure, and gRPC doesn't guarantee that the server is turned on when the client makes a request. Some expectations are unreasonable.

On the other hand, if you publish one set of proto files and all clients & servers consume these artifacts, the system as a whole is more "typesafe" and reliable than if you just post swagger docs and expect all your developers to check them daily for updates.

gRPC means API changes stand a good chance of getting caught by build systems. That seems about as reasonable a definition of "type safe across systems" as can be expected.

Couldn't have put it better myself.

I agree completely which is why I love proto.

> fast binary serialization/de-serialization with type safety

I'm not sure about Protobuf3, but certainly the older versions of protobuf could never be as fast as some other formats, because the message layouts were dynamic, meaning that you had to have lots of branches in the reading code.

> And the same applies to a typed RPC.

Erm. It clearly doesn't. Because with typed RPC the generated functions will be something like this:

    sendDetails(name: string, age: int);

And you'll get a compile error if you use the wrong type (assuming you are using a language with proper typing - which they are). With JSON you can't do that (without crazy hacks anyway).

nothing stops you sending bad data to a gRPC endpoint either

On the contrary, generating the client code using the same .proto definition as used by the server stops your client from sending bad data to the gRPC endpoint. Your binding code will fail to compile once the data definitions compiled from the .proto have changed. Then the client that sends bad data cannot be built.

They are talking about a work flow that prevents you from compiling bad clients. Of course if you don't use that workflow, you will still be able to make a bad client.

"Bad clients" in your example don't include malicious ones, who'll see a gRPC endpoint generating JSON to be consumed in a JavaScript app.

With no runtime type checking, JS's casting problems, and potential bugs caused by leaning on "type safe" serialization, there could be lots of black hat opportunities...

Protobuf/gRPC are not specific to any given language.

Well aware. I was answering mainly with regard to GP's quote, agreeing with IshKebab:

> There's nothing about REST+JSON that prevents type safety, as far as I'm aware.

Protocol Buffers can even serialize down to JSON (as does Thrift, for example).

Client code for PB+JSON has been a part of Google’s Closure library in fact (open source), and they used it in production for Gmail’s RPC, but I’m not sure if it’s still supported/recommended for PB 3.

The reason to use JSON instead of compact binary representation has been because JSON.parse is fast and doesn’t require any additional code. But nowadays JS is fast enough in most cases (unlike you’re doing something like streaming with media exts), but you can also ship Web Assembly/NACL library and (soon) delegate parsing to Web Worker with shared memory which should altogether give best possible performance.

gRPC has some benefits over REST. Since it's declarative, the entire API can be expressed as an RPC block in the .proto file, so your clients in multiple languages can be generated from this spec. With REST, you'd have to tell clients what protobuf structs to use with what endpoints. But of course the end result is roughly the same, once you've built clients and servers.

gRPC's built-in streaming and support for out-of-band data ("header" and "trailer" metadata) is also nice. For example, a gRPC server can emit trailing metadata containing an error, if something goes wrong halfway through a streaming request. Without this, you'd have to build a custom "protocol" on top of your Protobuf REST stuff; the stream would/could emit oneOf-structs that contain either payloads or metadata.

One downside to gRPC/Protobuf is that it's much less convenient to perform quick and dirty interactions against the servers. There's no standard, I think, for exporting the gRPC API to a client that doesn't already have the .proto file; I could be wrong, but don't think you can build a generic client that can talk to any gRPC server, list its available endpoints and so on.

Reply on news.ycombinator.com