Hacker News Reader

Comment by bitzun | Hacker News Reader

Unless you are doing bidirectional streaming (for which it seems pretty well suited, but I haven't used it, so it might be a fucking mess), grpc is usually a waste of time. Runtime transitive dependency hell, toolchain hell, and the teams inside Google that manage various implementations philosophically disagree on how basic features should work. Try exposing a grpc api to a team that doesn't use your language (particularly if they're using a language that isn't go, python or java, or is an old version of those.) Try exposing a grpc api to integrate with a cots product. Try exposing a grpc api to a browser. All will require a middleware layer.

View on HN

I've used grpc at multiple companies and teams within these companies, all of them 100-500ish engineering team size, and never had these dependency and tool chain issues. It was smooth sailing with grpc.

I have worked full time at now two companies of that size making the dependency and tool chain problems not be a problem for all the normies.

In my opinion, you shouldn't expose it to a browser, it's not what is good at, build something custom that converts to json. Like using REST to talk between backend services, makes no sense using a human readable protocol/api especially if there are performance requirements (not a call every now and then with a small amount of data returned).

To be fair, it was intended to be for browsers. But it was designed alongside the HTTP/2 spec, before browsers added HTTP/2 support, and they didn't anticipate that browsers wouldn't end up following the spec. So now it only works where you can rely on a spec-compliant HTTP/2 implementation.

The article seems to be an advert for this, with its plug of that hosted gRPC<->JSON service.

> Try exposing a grpc api to a browser

I remember being grilled for not creating "jsony" interfaces:

message Response { string id = 1; oneof sub { SubTypeOne sub_type_one = 2; SubTypeTwo sub_type_two = 3; } }

message SubTypeOne { string field = 1; }

message SubTypeTwo { }

In your current model you just don't have any fields in this subtype, but the response looked like this with our auto translator: { "id": "id", "sub_type_two": { } }

Functionally, it works, and code written for this will work if new fields appear. However, returning empty objects to signify the type of response is strange in the web world. But when you write the protobuf you might not notice

Bidirectional streaming is generally a bad idea for anything you’re going to want to run “at scale” for what it’s worth.

Why do you say that? I'm involved in the planning for bidi streaming for a product that supports over 200M monthly active users. I am genuinely curious what landmines we're about to step on.

bidi streaming screws with a whole bunch of assumptions you rely on in usual fault-tolerant software:

- there are multiple ways to retry - you can retry establishing the connection (e.g. say DNS resolution fails for a 30s window) _or_ you can retry establishing the stream

- your load-balancer needs to persist the stream to the backend; it can't just re-route per single HTTP request/response

- how long are your timeouts? if you don't receive a message for 1s, OK, the client can probably keep the stream open, but what if you don't receive a message for 30s? this percolates through the entire request path, generally in the form of "how do I detect when a service in the request path has failed"

> - there are multiple ways to retry - you can retry establishing the connection (e.g. say DNS resolution fails for a 30s window) _or_ you can retry establishing the stream

This isn't a difficult problem to solve. We apply both of those strategies depending on circumstances. We can even re-connect clients to the same backend after long disconnection periods to support upload resuming etc.

> - your load-balancer needs to persist the stream to the backend; it can't just re-route per single HTTP request/response

This applies whether the stream is uni- or bi-directional. We already have uni-directional streams working well at scale, so this is not a concern.

> - how long are your timeouts? if you don't receive a message for 1s, OK, the client can probably keep the stream open, but what if you don't receive a message for 30s? this percolates through the entire request path, generally in the form of "how do I detect when a service in the request path has failed"

We maintain streams for very long periods. Hours or days. Clients can detect dropped streams (we propagate errors in both directions, although AWS ALBs are causing problems here) and the client knows how to re-establish a connection. And again this applies whether streams are uni- or bi-directional.

> - there are multiple ways to retry - you can retry establishing the connection (e.g. say DNS resolution fails for a 30s window) _or_ you can retry establishing the stream

That's not how protobuf works? If a connection fails, you simply get an IO error instead of the next message. There is no machinery in gRPC that re-establishes connections.

You do need to handle timeouts and blocked connections, but that's a generic issue for any protocol.

Not going to give you any proper advice but rather a question to have an answer for. It's not unsolvable or even difficult but needs an answer at scale.

How do you scale horizontally?

User A connects to server A. User A's connection drops. User A reconnects to your endpoint. Did you have anything stateful you had to remember? Did they loadbalancer need to remember to reconnect user A to server A? What happens if the server dropped, how do you reconnect the user?

Now if your streaming is server to server over gRPC on your own internal backend then sure, build actors with message passing, you will probably need an orchestration layer (not k8s, that's for ifra, you need an orchestrator for your services probably written by you), for the same reason as above. What happens if Server A goes down but instead of User A it was Server B. The orchestrator acts as your load balancer would have but it just remembers who exists and who they need to speak to.

Nothing in Protobuf is suited for streaming. It's anti-streaming compared to almost any binary protocol you can imagine (unless you want to stream VHD, which would be a sad joke... for another time).

> Nothing in Protobuf is suited for streaming.

Uhh... Why? Protobuf supports streaming replies and requests. Do you mean that you need to know the message size in advance?

No, Protobuf doesn't support streaming.

Streaming means that it's possible to process the payload in small chunks, preferably of fixed size. Here are some examples of formats that can be considered streaming:

* IP protocol. Comes in uniformly sized chunks, payload doesn't have a concept of "headers". Doesn't even have to come in any particular order (which might be both a curse and a blessing for streaming).

* MP4 format. Comes in frames, not necessarily uniformly sized, but more-or-less uniform (the payload size will vary based on compression outcome, but will generally be within certain size). However, it has a concept of "headers", so must be streamed from a certain position onward. There's no way to jump into the middle and start streaming from there. If the "header" was lost, it's not possible to resume.

* Sun RPC, specifically the part that's used in NFS. Payload is wildly variable in size and function, but when it comes to transferring large files, it still can be streamed. Reordering is possible to a degree, but the client / server need to keep count of messages received, also are able to resume with minimal re-negotiation (not all data needs to be re-synced in order to resume).

Protobuf, in principle, cannot be processed unless the entire message has been received (because, by design, the keys in messages don't have to be unique, and the last one wins). Messages are hierarchical, so, there's no way to split them into fixed or near-fixed size chunks. Metadata must be communicated separately, ahead of time, otherwise sides have no idea what's being sent. So, it's not possible to resume reading the message if the preceding data was lost.

It's almost literally the collection of all things you don't want to have in a streaming format. It's like picking a strainer with the largest holes to make soup. Hard to think about a worse tool for the job.

Ah, you're an LLM.

Protobuf supports streaming just fine. Simply create a message type representing a small chunk of data and return a stream of them from a service method.

> Try exposing a grpc api to a team that doesn't use your language

Because of poor HTTP/2 support in those languages? Otherwise, it's not much more than just a run of the mill "Web API", albeit with some standardization around things like routing and headers instead of the randomly made up ones you will find in a bespoke "Look ma, I can send JSON with a web server" API. That standardization should only make implementing a client easier.

If HTTP/2 support is poor, then yeah, you will be in for a world of hurt. Which is also the browser problem with no major browser (and maybe no browser in existence) ever ending up supporting HTTP/2 in full.

Reply on news.ycombinator.com