Hacker News Reader

Comment by ramesh31 | Hacker News Reader

Yeah imagine if instead of just using plain text JSON for your web application you could use a proprietary binary format that web browsers know nothing about and require a translation layer to talk to. It's awesome.

View on HN

There's no reason you couldn't use gRPC with json as a serialized message format. For example grpc-gateway [0] provides a very effective way of mapping a gRPC concept to HTTP/JSON. The thing is, after moving to gRPC, I've never really felt a desire to move back to JSON. While it may be correct to say "parsing json is fast enough" it's important to note that there's a "for most use cases" after that. Parsing protos is fast enough for even more use cases. You also get streams which are amazing for APIs where you have to sync some large amounts of data (listing large collections from a DB for example) across two services.

With gRPC you also have a standardized middleware API that is implemented for "all" languages. The concepts cleanly map across multiple languages and types are mostly solved for you.

Adding to that you can easily define some conventions for a proto and make amazing libraries for your team. At a previous job I made this: https://github.com/CaperAi/pronto/

Made it super easy to prototype multiple services as if you mock a service backed by memory we could plop it into a DB with zero effort.

I think this "gRPC vs X" method of thinking isn't appropriate here because protos are more like a Object.prototype in JavaScript. They're a template for what you're sending. If you have the Message you want to send you can serialize that to JSON or read from JSON or XML or another propriety format and automatically get a host of cool features (pretty printing, serialization to text/binary, sending over the network, etc).

[0] - https://github.com/grpc-ecosystem/grpc-gateway

> The thing is, after moving to gRPC, I've never really felt a desire to move back to JSON.

Second this. I think its also really important to consider the "trap" of going in on gRPC, but using something like grpc-gateway to also spit out JSON as a "backup". We did this for a project I was on, and the JSON API was the thing everyone else used (because up until then, everything there was a JSON API, naturally). As a result, unless a consuming team was willing/able to get involved in the protobuf definition internals, most of our consumers didn't reap any benefits from our typed protobuf API.

I know it really isn't hard to do, but lowest friction denominator wins far too often in a feature-focused environment :-\

One thing that was doable in my last company where I used gRPC was I made interfacing with the gRPC API dead simple. Something like `connect(ServiceBlockingStub.class)` was in your code and it would instantly pull configs, build the service, wire up all of our middle wares + logging, etc.

You can automate a lot of this which essentially makes the normal level of friction easier.

That's pretty cool. Was there a particular local environment needed to make this work? I recall auto generation being a part of the whole suite, but sounds like you took it up a notch.

The special sauce was establishing standards that were used across the code base that we could detect. For example, all configs were from environment variables in the form of `<ServiceName>_<DestinationName>_URL` or something. If the foobar services wants to talk to the auth service it would read the environment variable `FOOBAR_AUTH_URL`. Then, from conventions like this you can build libraries/tools and layer them on to simplify.

The way you write your deployments can also know about this convention and automatically generate the environment variables you need (didn't get to this point). I hope one day I get a chance to work on building some tooling like this, open sourcing it, and making it easy for others to adopt a similar mentality.

Why are you using a browser to test microservices, for which browsers are an atypical-at-best client?

Amiga got right what Unix got wrong: A well-structured, open binary protocol is almost always preferable to a "plain text" protocol because the parsing overhead really adds up.

Try telling that to a platform with a different endian format.

Little endian won. If you're developing modern software for modern machines, you pretty much needn't even consider big endian.

What does this (client to web service communication) has to do with interservice communication?

You still have to unmarshal json into some sort of object/struct.

Agreed. Use JSON, proprietary binary formats are terrible, and 99% of the time the bandwidth savings aren't worth it. For messaging, I really like NATS (incubating in the K8S landscape - probably graduating soon). My fav thing is the wildcard usage in NATS.

Unless you are communicating C++ services for example and you prefer a lib that does the parsing and transformation to native object for you

Bandwidth isn't the primary concern here. CPU time spent parsing/unparsing is. Depending on how your application is structured, it can actually dominate time spent in application logic. Whatever the case, it's simply waste heat being added to the universe. Use a little-endian, 64-bit-aligned binary format, and parsing overhead simply goes away.

Great point. I'm not one to over optimize, but seems like parsing messages for internet sized apps is worth spending a little effort to save energy and environment.

Please elaborate on the ways in which protocol buffers conforms to the definition of the word "proprietary".

Reply on news.ycombinator.com