Show HN: Pbkit – Protobuf toolkit written in Deno/TypeScript | Hacker News Reader

Show HN: Pbkit – Protobuf toolkit written in Deno/TypeScript

pbkit.dev

69 pointsdisjukr13 comments

This looks very interesting! Anything that can move people away from protobuf.js (which seems to no longer be maintained and depends on prototype values for "default values" meaning that you can't send deserialized protobuf messages to/from web workers) and the "native" JS codegen by Google (which produces code that is both very slow and a awkward to use) is a win in my book.

We're currently using https://github.com/timostamm/protobuf-ts which has been fantastic. Its codegen is dependent on the protoc binary as it is implemented as a protoc plugin, but the code it generates passes the protobuf conformance tests. The generated code also outputs plain objects when deserializing protobuf messages which means it works perfectly when sending stuff to/from web workers. It also has grpc, grpc-web, and twirp clients.

If you are considering protobuf for browser <-> server communication (as opposed to server <-> server) a good alternative is Twirp [1] by twitch. Like gRPC it uses protobuf for API schema, but unlike gRPC it works out of the box with HTTP 1 without needing any proxies.

I found both the golang server integration and the typescript client generated to be idiomatic and easy to use. There are community integrations with other languages too - but ymmv.

[1] https://twitchtv.github.io/twirp

Just be aware of the lack of streams: https://github.com/twitchtv/twirp/issues/3

Any advantages to gRPC-web? We've been using it for a while and it's a bit clunky, but it works.

We stick envoy in the middle for gRPC-web <-> gRPC; what do I need for Twirp?

It works oob - the browser can directly talk to twirp service.

I used it because I basically wanted end-to-end type-safety without any infrastructure overhead and additional decode-encode hops in betweeen.

Ah, but it's not gRPC compatible then; just very similar and uses the same proto syntax for services and messages?

Yes, its an alternative.

protobuf is a lower level specification that supports defining typed services. It does not dictate things like transport mechanism or other implementation details.

There are multiple protobuf compatible RPC implementations, gRPC obviously being the most popular one, twirp being another.

Our company uses gRPC/Protobuf intensively.

I'm a web front-end engineer, and one day my boss informed the client engineers that the API provided by the back-end would be in the form of gRPC only.

To use the gRPC/Protobuf stack in the web frontend, there were two options: protoc and Protobuf.js. Protoc generated JavaScript code old-schooled and required native dependencies, and Protobuf.js was problematic for us at our scale.

And we are a company that makes mobile apps that use webviews a lot. When webviews communicate with natives, they communicate using the interface defined by the protobuf service schema. The service code generated by the existing protobuf tooling is tightly coupled with gRPC, and it was necessary to make this part into a structure that can be easily replaced.

So we started making our own protobuf compiler, and while building the compiler, we also made a package manager, VSCode extension, and Chrome Devtools.

As far as I know, there is no tool that properly supports functions like Go to Definiton in VSCode extension. We have implemented this function properly, and we plan to implement other functions using our language server such as autocomplete someday.

Now we've stripped out all of the Protobuf.js code, which was a big part of our biggest product, and rewritten it with Pbkit.

There is another pretty good solution for this that will allow your front end to stick with JSON and keep your backend as gRPC only which is to stick envoy in front of the gRPC services and use a filter to translate between the two which is what you are already doing except you are doing gRPC-web to gRPC right now since gRPC in the browser isn’t currently possible but that should be changing hopefully before too long with technologies like WebTransport (https://web.dev/webtransport/).

I’m only familiar with Google’s cloud stack but they have basically managed versions of this already in the form of their Cloud Endpoints and API Gateway products that will handle all of this for you out of the box. I can’t speak to AWS or others but like I said at the end of the day it’s really just an Envoy filter doing the heavy lifting.

But in short you can 100% write gRPC only backends but still expose a JSON / REST API and nobody has to know anything to do with gRPC and you can just wait for the browser technologies to catch up in the meantime and make the transition when you’re ready.

Nice.

We are using (https://github.com/whisklabs/grpc-ts)[whisklabs/grpc-ts]. We saved %80 of our bundle size by changing from grpc-web. Those generated proto-js files were big!

Broken formatting. Link is https://github.com/whisklabs/grpc-ts

Would there be advantages in writing this in web assembly with a TypeScript layer, or would the IPC be to costly?

its good to see all cool projects that are written in deno and ts

Reply on news.ycombinator.com