Hacker News Reader

Hacker News

Comment by spenczar5 | Hacker News Reader

Full thread

spenczar5

Hey everyone! I'm the OP and primary author of Twirp. I'm happy to answer any questions and hear feedback.

You can also reach me directly, if you like, email is in my profile.

View on HN

I like the simplicity of the library. I haven't tried it yet, but this behavior is a bit unfortunate choice

> If there is an error, it will be JSON encoded, including a message and a standardized error code.

Why not to return standard protobuf error when the source was a protobuf? It massively complicates things when you have to expect errors in one format and responses in the other.

It’s important for errors to be human-readable off the wire. If they were proto-encoded, you couldn’t get at them easily with tcpdump, and you couldn’t read erroring response bodies from curl, and you couldn’t read errors shipped to stuff like rollbar.

Hopefully, the complexity is encapsulated by the generated client.

What about allowing/respecting an Accept header in the Request? In @doh's case, if the client only specified Accept: application/protobuf that would override the default behavior of returning JSON encoded errors.

That’s a pretty good idea. It does expand the complexity of the client a bit, but at least it’s in an opt-in way so it doesn’t strictly need to be done for cross-language clients.

But what would the benefit(s) to users be? If they are deserializing a protobuf error, they are almost certainly using a generated client, so I don’t think they will know or care how the error was encoded.

(This might be better as a github issue to keep a visible record of the design for others.)

I think it depends who the user is in your case. In mine it's the developer who has to work with Twirp outside of the standard libraries (maybe different language, maybe just wants to incorporate it in their own code, ...).

I also like when things are consistent without surprising behavior.

Did you ever consider Cap'N Proto [1] as well? How does it compare? It looks like it could easily be integrated as a third transport encoding besides JSON and ProtoBuf.

[1] https://capnproto.org/

Yeah, we looked at it a bit. To be honest, the biggest blocker was that we wrote Twirp as a gRPC alternative, so we already had protobuf service definitions in some spots, and it seemed easiest to keep using the same thing.

Adding another transport encoding would be good, but its important that any Twirp server can support all of the encodings. We would need to be able to map a message defined in capnproto to a protobuf message definition, which didn't seem completely trivial when we looked at it, since capnproto uses its own IDL, I believe.

I don't know a ton about capnproto, though, and I'm open to learning more about it. We would just need to work under the constraints that JSON and Protobuf requests would need to still work.

What do you think about prpc, which is our version of "gRPC for http/1.1" https://godoc.org/go.chromium.org/luci/grpc/prpc

I hadn't even heard of it until yesterday, to be honest. I know very little about it. It looks like it does a good job addressing the HTTP 1.1 concern, for sure, but I don't know whether it addresses the other issues we've had. I'd have to spend a lot more time reading to understand it.

Thanks for posting! Does Twirp use a single HTTP request per call, or does it use a persistent connection similar to the net/rpc package? If the latter, does it provide options for heartbeat, reconnect and retry?

EDIT: the blog post covers this in the protocol section. Every request is a POST :)

It's one HTTP request per call, but requests usually flow over a persistent connection in HTTP 1.1 - it's certainly not re-opening a connection for every request.

TCP reconnection and stuff like that is at a lower level than Twirp. When you make a client of a Twirp service, the constructor accepts a http.Client which it'll use to send the requests. http.Client has a "Transport" field which is responsible for opening connections, that sort of thing. The Go standard library's defaults are pretty good, but you can tune it as you like.

The blog post mentions the following:

> The core design of Twirp is language-agnostic and we’re planning to expand into new languages, but our Go implementation is already stable and capable of serving heavy production loads.

I was wondering, is Rust on the roadmap?

It's not on the roadmap because I have zero experience in Rust. I'd be very happy to see the community make a Rust generator and could provide guidance on getting the protocol side right, but I wouldn't be able to tell whether the generated client was idiomatic - which is really important.

It'll take someone who is very fluent in Rust and who is motivated to do it, but I'm all for it.

Thanks for posting! How did you work around issues with GRPC on AWS/ELB?

We tried grpc-gateway a little bit. But mostly, we wrote Twirp as a gRPC alternative because it was hard to work with ELBs.

We had to stick to layer 4/TCP via an ELB (as opposed to an ALB).

gRPC connections are persistent, so a single client will always talk to a single backend in that configuration. It's fine if you have lots of clients but can easily lead to load imbalances.

That's why projects such as Envoy exist. I'd link it, but I'm on mobile.

Keep an eye on it.

You can use round robin load balancing in gRPC without Envoy.

And it's not difficult to swap it out for a consistent hash balancer or other solution.

You are doing L4 load balancing with the ELB? So you can’t do sticky session management? Is that a problem you run into, if so, how did you handle it?

When I implemented GRPC on ELB, we used a multiplexer to re-route the GRPC requests. See: https://github.com/soheilhy/cmux/. The only other issue we had was that ELB's would not let connections live longer than 30 seconds.

You can increase that connection timeout, although there's an upper limit of an hour or so which it will eventually exceed.

I completely forgot this. Makes it a lot more feasible to use it.

What works for us (grpc)

Elb with layer 4 and proxy protocol enabled. Behind Elb sits nghttpx (not nginx) doing TLS termination and request forwarding to gRPC.

Proxy protocol is used to keep the source IP.

This setup is all done with Kubernetes using kops for the cluster setup, nghttpx-ingress-lb as the ingress controller. Also we have multiple namespaces/environments in Kubernetes (staging/dev..), so nghttpx does routing based on the hostname.

We tried linkerd before but somehow failed using it as an ingress controller doing TLS termination and upstream HTTP2. Doing the other way of routing everything through a dedicated linkerd port and a dtab worked, but mixing in TLS termination + upstream HTTP2 in a single dtab stopped us.

So for now we keep this simpler setup and we probably are going to check out Istio/Heptio/Envoy

Reply on news.ycombinator.com