Comment by losvedir | Hacker News Reader

Comment by losvedir | Hacker News Reader

Parent Full thread

Interesting. I wonder to what extent it's found use at Google over this past decade.

There are advantage to being language-specific, but a lot of disadvantages, as well (speaking as someone who recently had to write some Elixir code to unmarshal a Ruby object...). It seems hard to introduce this since you're forcing all communicating services to be Go-based, which is kind of contrary to the independence that microservices usually affords you.

Some of the benefits are simply design goals (e.g., top level arrays) which could also be done in a language-independent protocol. And even performance questions probably could. Like, Cap'n Proto I think is designed so that users of the protocol don't have to serialize/deserialize the data, right? They just pass it around and work with it directly.

I can see Rob Pike being frustrated with Protocol Buffers at Google, and I don't begrudge anyone for taking a big shot like this, but I wonder if he's found any success with it.

Yeah, after years of dealing with language-specific serialization formats---and inadvertently learning internals of them (including Go gob, Python pickle and PHP serialize), I'm over. And gob is not even a schematic serialization format (i.e. not only you don't need to define a schema beforehand, you can't). There is some interesting idea, but that's all. Use a well-known schemaless serialization format with some extensibility [1] if you really need.

[1] Maybe there was no suitable one when Go was first created. Nowadays I believe CBOR is the best format for this job.

I'm in the same boat. Not to mention security concerns that often crop up in (interpreted) language specific deserialization (I'm looking at you, pickle, thinly veiled `eval()`). I agree that CBOR should generally be the serialization tool of choice for self-describing data (ie. in places where you might otherwise choose JSON).

And if your language of choice doesn't have a CBOR lib, CBOR is fairly easy to implement and writing a encoder/decoder is very fun! I recently completed my implementation for the Gerbil Scheme language last week [0].

[0]: https://github.com/chiefnoah/gerbil-cbor

Yep, I very much agree with this. It's probably inevitable that languages with reflection grow some kind of language-specific serialization format because a) they can and b) there's often some use case where it looks handy to have a format adapted to the quirks of the language. Plus, when some of these bespoke formats were created, the world hadn't converged as much as it has now around a few common text and binary formats.

But now the interoperable serialization formats have a lot more energy spent on tools and such, and the formats are better-specified, to the point that you probably want to use them even where interoperability with other stuff doesn't force it.

As an engineer at Google, my opinion on Protocol Buffers changed massively. Pre-Google I found them awkward, the language bindings in Python sucked, and I didn't really see the point. I knew a schema for services was a good idea, but protobufs didn't seem like the best option.

The thing is that at Google protobufs are used everywhere. Like, absolutely everywhere. Think of all the places they could be, and it's way more. All the tooling understands them, code search and go-to-reference works on them everywhere, they are truly transformational on how many different services with many different implementations interact.

Are they perfect? Far from it. But a Go-only implementation misses almost all the value of protobufs. If I was inventing them for Python I could do better (pickle? maybe not) but the whole point is that they aren't language/ecosystem/use-case specific. If this was Rob Pike's frustration then I can't help but feel he missed the point, or this post is a little disingenuous as to the benefits.

I've not seen Gobs used in Google, but I'd imagine an engineer would need to make a very strong case against using protos between services, regardless of if both services are in Go.

It's called "Larry & Sergey Protobuf Moving Co." for a reason. See the t-shirt design in the video screenshot here: https://isocpp.org/blog/2020/07/cppcon-2019-there-are-no-zer...

Disclaimer: an employee at the said moving company.

In my opinion this is pretty on-brand for Pike.

Back when he did his best work, it was possible for one person to "just write the new thing", without making it fit with anything else. There was nothing else to fit it with. You could invent everything from scratch, and not only was it not a waste of time, if you were good enough it had a chance of being the best fit for purpose.

You could take shortcuts. You could have every part of your system be "odd", because nothing was "even".

That's not true anymore. And the way I see it Pike has not moved on.

Science in general had this switch at one point, too. There was a point where one could know all of science. But it's long gone.

This is an unnecessary ad hominem attack. He wrote this over a decade ago, in the midst of doing his best work creating Go itself.

gob was his opinionated way of doing a Go-specific encoding, while also supporting any number of other encodings in the language. Go has incredibly good support for almost every popular encoding there is.

gob has also been used successfully by a number of projects. In many cases, it's a perfectly good way to encode a piece of data that is completely local to a Go program.

> This is an unnecessary ad hominem attack

I didn't mean it as such.

> in the midst of doing his best work creating Go itself.

Not to go too far off topic, but Go is another example. It famously ignores decades of language theory, and they wrote their own assembler, linker, etc.

Now, much of that has been undone and rewritten, as Go became more adopted, requiring playing well with the rest of the ecosystem.

(but much of it we're unfortunately stuck with, because it's part of the language)

30 years ago there was no ecosystem to play well with, and compared to now we were just banging rocks together. Back then you could be a CS polymath as one person. Well, I couldn't, but Pike could.

It was the old days of John Carmack starting every game engine with an empty directory.

I'm saying that today nobody can. Even John Carmack could not on his own write a AAA game. (I know ID had other coders, but my point stands)

Well, to go off-topic with you, the idea that Go "ignores decades of language theory" is just an opinion that you hold. A factual statement would be to say that the Go designers omitted a great many features other languages have included.

The idea that they did so out of ignorance is ridiculous, given the background of the Go design team. They made considered decisions about what to include in Go.

A fixed-gear bicycle is not "ignoring" decades of bicycle design theory.

Creating a programming language is an engineering task not a theoretical task. Which means there are major trade-offs to be made. And they chose their trade-offs. The wild success of Go should at least make you consider whether or not they're better at making these trade-offs than you, and most language designers, are.

Maybe they were wrong about some choices, which is why Go is still evolving, but they were self-evidently mostly correct.

Most of the Reddit/HN/Twitter crowd that are parroting the "Go ignores decades of language theory" line are just parroting something that originated in an incredibly toxic part of the Scala community. The vast majority of them don't actually understand the tradeoffs and implementation complexity that are associated with the specific subset of language features they desire. The debate about a particular feature is often not even concluded in languages that include said feature. For example the Swift team still has serious disagreements about generics and their respective implementation.

I've seen Pike have conversations about language design with SPJ, Hejlsberg, Lattner, Stroustrup, Odersky and other highly respective PL designers and they would never make such a shallow and trite comment.

Well, that's your opinion.

You seem to have gotten a bit emotional about this, so I don't think this'll go anywhere.

The comment you're replying to is calm, reasoned and courteous.

Claiming the poster is being emotional is... poor form, to put it mildly

Oh. I found it very defensive and aggressive.

Putting scare quotes around what I said, calling what I said "ridiculous", and seems to have taken critique of the Go language personally by saying that the Go language designers better than most language designers.

So this is why I found the reply neither calm nor courteous. As for reasoned, I don't see any reasoning it it, just conclusions.

I don't even see any sign of a refutation of my point, which isn't so much about if Go is good or not, but about the needless reinventing and ignoring other work, and as a result running into problems that would have been predictable, had they been taken into account.

To me it summarizes as "they're smart, you're dumb".

If this is the type of replies this person would write, then that's not productive.

> and they wrote their own assembler, linker, etc.

True, but over 30 years ago. In fact, said compiler was the very first program written for Plan 9. Those decades you speak of came afterwards.

Interesting hypothesis. I can't comment on Pike's history here, but at Google there's certainly a noticeable difference between old special cases that were built pre-2015 ish, and the modern world where everything is very cohesive. I get the impression that there was a big push to achieve that, led from around that time by various products. It's hit some areas more than others, but there do seem to be almost no new products building in the way you've described now.

Well, it is 2011 post. Now Pike is retired, I don't think he cares or matters anyway.

I think he matters, for computer history. I never aimed to have him care what I say, though.

As awkward as Protobufs might be - is there any similar format with so many client bindings? I tried comparing the message formats I could find (e.g. Cap'n'Proto, Protobuf, msgpack and others), and still Protobuf seems to have the most supported languages. I'd be happy for any other suggestions!

> I can't help but feel he missed the point

Did he miss the point, or was the point to light a fire under the protobuf team to improve their product? Which they eventually did (e.g. required fields were removed after this post was made).

FYI: required fields were re-introduced in protobuf. why? because despite what the gobs team and others thinks, that is a necessary feature to work with data structures. If "everything is optional", what is the value of any structure, over sending an untyped associative array of stuff?

"Optional" fields, as in fields you can tell if they were explicitly set or not, were re-introduced. Depending on your definition either required fields were not reintroduced, or required fields with a mandatory default value were until recently the only kind of field in proto3.

It's possible that was an aim, and I'm not familiar with the timelines here, however this does again focus more on the mechanics of protos which I still don't think matter as much as them being fully integrated into a company like they are at Google. The mechanics of them are still not great. Maybe this was a blocker to that happening though?

Nice recap.

By the way, just curious what is your opinion on MessagePack?

Reply on news.ycombinator.com