"The solution is as follows:
Make all fields in a message required. This makes messages product types."
Except it also breaks backwards compatibility, one of the most powerful and sought-after features of protobufs.
Make all fields in a message required. This makes messages product types."
Except it also breaks backwards compatibility, one of the most powerful and sought-after features of protobufs.
It doesn't have to. Just add row types to handle unknown content, ie. if an intermediary knows only of fields foo and bar, then they can process any data with such fields if given a type like "type SomeRecord = { foo : int, bar : string | r }", where 'r' represents the remainder of the record.
The article's criticisms are valid and there are typed solutions to most of the objections that have been raised against it.
You absolutely could in multiple ways:
1. You make every accepted product type have a row type at your service interface if you expect schema evolution.
2. If you have to add a field unexpectedly, ie. where you did not have a row type, then you must deprecate the old API. If this seems onerous to you, then your service infrastructure is probably insufficiently flexible.
Option 2... look. I've seen a lot of API deprecations, across multiple teams in multiple companies, and every one of them was very onerous in ways that had little to do with the service infrastructure. If you've done easy API deprecations, more power to you, but I don't think your experience is representative.
The problem with making fields required is that older serialized protocol buffers parsed by newer message definitions may be missing newly added required fields, which will break things.
In other words, proto has a typed interface, but you must runtime check that a given bag of bytes conforms to that typed interface.
This is true for any io.
I assume you mean serialised data, not deserialized. And yes, deserializing includes type checking. The point is that this happens once and the need for a separate API for dynamic data shouldn't be needed.
The data under discussion isn't "dynamic", it's still static, it just isn't known to the schema in question at runtime (since it's only known to a different schema). That means you can't access it by name, since the field names aren't known.