Gunk: Modern front end and syntax for Protocol Buffers | Hacker News Reader

Gunk: Modern front end and syntax for Protocol Buffers

github.com

73 pointskenshaw39 comments

I'm not seeing what makes this "modern". proto3 is only a few years old and nothing about it strikes me as unusually archaic. Protobuf in general isn't that much older than Go. I can see why Go-compatible syntax would be attractive to Go developers, so maybe that should be in the description rather than "modern"?

There is a semi-consensus that Proto3 was created to be compatible as Go basic types. Making Go types from Protos, or Proto types from Go is what is being entertained as modern.

Proto3 has features (discussed elsewhere in these comments, like oneOf) that aren't present in Go's type system. I'm not clear on why constraining it so it's more narrowly compatible with a particular implementing language is more modern.

It would be useful to see an example of the workflow of using this tool compared with the equivalent workflow while using protoc. That would be a selling point, while "modern" is just a sticking point.

The compatibility is not just with Go types, but any C structure compatible types in any language.

Not sure why I’d want to define a language independent interchange format in a language specific way and remove all of the tooling help at the same time. Why is this better? A why section/motivations would help greatly.

There is an explanation in the README file, but I'll explain:

It helps to standardize workflows across a number of different source repositories (all Go based), where a number of different developers had come up with different ad-hoc ways of generating code against protobuf definitions. Since the syntax is Go-derived, it reduces cognitive load for developers when specifying protocol buffer definitions. We understand that it may not be for everyone, but it works for our team.

Why not just make a simple go tool that just covers that issue and still uses proto thus getting the tooling and expressiveness instead.

I'd be interested to see a "why?" section in your docs. In general I think making new "skins" for the same underlying tech is not a good thing. It fragments the ecosystem without any real business purpose. Is there some problem this addresses that is not already solved equally well by proto2 or proto3, besides that it doesn't look like a particular programming language?

Yes, this has the added benefit of standardizing protocol buffer based workflows across teams / repositories. I appreciate the criticism, but we (the Brankas developers) find this syntax significantly easier to use, as well as the tooling provides a better mechanism for using other protoc generators.

We just pushed the first initial, public release of Gunk, a modern frontend and syntax for Protocol Buffers.

Check out releases here: https://github.com/gunk/gunk/releases/tag/v0.1.0

Please note this is still extremely early / alpha stage, but we are soliciting feedback on our initial design. We hope to make this project useful to the greater developer community.

Protobuf has first class enum support and also provides a thing called "oneof". Go doesn't have anything of this. So, it is clearly not the right tool for the job. Sorry guys, you are wasting your time sticking to Go constructs only.

Thanks for the input -- this actually does support enums, using a variant of Go's const:

    type MyEnum int

    const (
        MYENUMVAL MyEnum = iota
    )

It's a philosophical argument, but one should not design a protocol using "oneof". I believe that's a relic from proto2, and I'm not aware of any modern APIs published by Google, that make use of "oneof" in anything but a legacy capacity. Gunk is not for old protobufs, it's for new development. It works with and integrates cleanly with existing protobuf definitions.

I was curious about this claim, so I searched for oneof in the googleapis repo[0][1]. There's a bunch of results, all in proto3, and many were committed for the first time (at least publicly) in the last 4-6 months. So I don't believe this is true.

From a safety perspective, tagged union types, which oneofs are, are incredibly useful and really should be used more often.

[0]: https://github.com/googleapis/googleapis/search?q=oneof&unsc...

[1]: I have no idea what this repo actually does, to be honest, but it appears authoritative about something, and appears to be public grpc apis Google exposes.

In proto3 all fields are optional. As such, I don't see the need for "oneof" since you should check in business logic whether or not the field is there. By using "oneof" you are declaring two different types can be the same name. It's a philosophical argument that you're free to disagree with.

In good proto2, all fields are optional, that doesn't negate the uses of one of, which provide compile time validation instead of runtime, which is the important part.

Yes, I understand how this would be done. It's a design philosophy. If you'd like Gunk to support "oneof", we're open to Pull Requests!

I just can’t get my head around this reasoning, it’s just as valid to ask for/return a “foo OR bar” as it is to ask for a “foo AND bar”. Any protocol that rejects that pushes that complexity elsewhere, somewhere less explicit.

I feel very strongly that type safety is of paramount importance in development, and "oneof" (and other similar constructs) in protocol design breaks fundamental ideas of type safety. It's a feature of the IDL that doesn't translate well to the actual implementing language, is not strictly necessary as the same thing can be accomplished through other designs that have an added benefit of being more explicit, both to the implementer and to the consumer.

No, "oneof" increases type safety. "oneof" describes mutually exclusive fields. For example:

  message Filter {
    oneof kind {
      Tags tags = 1;
      IPRange ipRange = 2;
    }
  }

With this in place, a client has to check the type of the filter in order to access the actual filter:

  switch t := f.Kind.(type) {
    case *Tags:
      // ...
    case *IPRange:
      // ...
  }

Without it, you can get into a situation where it's possible to create invalid combinations:

  message Filter {
    string type = 1;
    Tags tags = 2;
    IPRange ipRange = 3;
  }

Now you can accidentally end up with code like this:

  f = Filter{}
  f.Type = FilterType_Tags
  f.IPRange = IPRange{}

or:

  if f.Type == FilterType_Tags {
    useFilter(f.IPRange)
  }

You believe in type safety yet refuse to let an IDL dictate a type at compile time?

"oneof" construction is here for two reasons:

1. It enhances readability 2. It would be a surprise to you: oneof actually enhances type safety. It is just lacking type system of Go where you cannot express the feature and thus loses type safety.

Let me guess: you are talking about types and type safety without learning type theory and type algebra. You are just throwing out 1 of the two basic building blocks of type algebra and sticking to your point even though everybody tells you how important it is. Anyways it's ok if you want to be Go specific, this is totally fine.

Type systems without sums in 2019 is ignoring a lot of history and evidence, new and old.

Oneof exists in proto3, so a tool that doesn't enforce it is likely to create messages that don't conform to the schema. The style guide also doesn't discourage it.

https://developers.google.com/protocol-buffers/docs/proto3#o...

Gunk is a frontend to protoc. If one needs to use "oneof", it's still possible to just define it in a .proto file. Gunk interoperates cleanly with both .proto and .gunk files.

"Oneof" is certainly not a relic of proto2.

I'm curious, in your view, why should protocols not use "oneOf"?

Not GP commenter, but a lot of the decisions to remove explicit optionality and requiredness from protocol buffer is that it made pb too complicated to change when you wanted to make a field required or optional, so they punted with 3 and said "do this in service logic"

I think the feeling is that this is the realm of business logic, and a protocol should be simple and fast and not concern itself with the rules part of the underlying service.

Oneof seems to fall under the same concept, where it says "one and only one field from this list" when they can just as easily do that in service or client logic (three fields, documentation and a server response custom written as to the WHY of oneof relationship)

I can't follow that argument, as it seems to apply just as much to wanting to change the type of a field, the name of a field or just making changes in general. Maybe I deal with relational databases too much, but I really miss being able to declare fields required and provide default values, because now instead I have to document, trust others will read the documentation, and provide explicit error handling for when constraints that used to be validated client side or server side by protobuf. I also miss not having a NULL value, but protobuf never had that and almost certainly due to dealing with too many relational databases ;)

If I lose oneof too, I might as well be using JSON.

Your points are valid, but ultimately an API is defined by its semantics in addition to data types. For anything beyond extremely primative RPC, you have to read the docs anyways.

Moving to proto3 means putting all your validation in one place, your serialization primitives in one place, and your docs in one place, instead of smattering them all over. It's a lot less error prone and way more maintenance friendly.

Why is `// required, the server will respond with a failure if this argument isn't provided` in the .proto file enough documentation? That's usually what I go to.

lowmagnet more or gave a succinct summary. I've also now added other comments here, discussing my views. I don't believe it's necessary, and I would proscribe other ways to accomplish the same design. It's a philosophy of API design, not some kind of absolute stricture.

oneof was NEW in proto3 as far as I know, not a relic of proto2. I've ONLY used proto2 and I've never used oneof.

That isn't an enum because it's valid to do SOME_NON_ENUM_VALUE = MyEnum(10) in Go.

This is pretty cool. Have you all looked at the annotations to build pb in kubernetes? They use proto2 so they can use optionality, but they use a lot of annotations to write the pb like this project.

I haven't specifically looked at the Kubernetes code base, but I will -- thanks for the pointer! We plan to expand / support / better standardize the annotations in the github.com/gunk/opt package in time.

It would be good to explain (perhaps in a blog post) why this is useful. It's not obvious to me.

Neither the syntax nor build process for protocol buffers has ever seemed particularly difficult.

You mentioned something about cognitive load, but even as a Go developer this looks like it would add to my cognitive load. I have to learn/remember this new syntax and learn/remember the (imperfect) ways in which it maps to protocol buffers, etc.

I could really use some concrete examples of the ways this is better.

I don’t understand the purpose. The existing proto syntax is both easy to learn and language agnostic.

I’ve watched new devs pick it up in a matter of minutes.

"Modern"?

Reply on news.ycombinator.com