Hacker News Reader

Comment by saberience | Hacker News Reader

You don't work for Blizzard do you? When I worked there engineers (not my team thankfully) designed their own alternative to protobuf to try and save more bytes over the wire, which in my opinion was a really poor decision. Rather than get on with actually adding value, they ended up pushing back multiple other teams deadlines while adding almost no value. It was a classic "not built here" mentality and doing engineering work for the sake of the work, rather than actually ask "what problem are we trying to solve here?"

If protobuf works for Google then it essentially works for 99.999% of every other company on the globe.

View on HN

If protobuf works for Google then it essentially works for 99.999% of every other company on the globe.

I can't agree with this mindset.

Another commenter here pointed out the Google implementation is 20x slower than others, and 1.6MB for this kind of task feels bloaty. Just because it meets Google's needs doesn't mean it's universally adequate.

Why exactly does a megabyte matter, unless you're trying to run on a 90's watch or embedded?

Maybe because "20X slower"

These are simply tradeoffs. For example, one can make very fast protobuf parsers/serializers if you prioritize that over the ergonomics of the generated code.

Well, transmitting a megabyte also takes a full second at 8 Mbps.

Nobody's transmitting this, it's the size of the library doing the proto serialization/deser

I did a search on this page and I didn't find the comment you are referencing. It seems very unlikely to me that google's implementation is 20x slower.

See tijsvd's comment about 1 microsecond vs. 50 nanoseconds. (Perhaps I should have caveated that with "in some scenarios")

Which implementation of protobuf is 20x faster than Google's?

I think the general point is 'if it works for Google, you should feel relatively safe adopting it for your own use'. It doesn't mean it'll fit every single usecase.

If what you need is something that is super efficient over the wire - or any other such requirement not filled by Protobuf - then maybe look for a different protocol altogether. Or design your own!

But for the rest of us - the '99.999%' - the trade-offs are well understood and we'd rather go for the tool we know than reinvent the wheel or use a less maintained tool.

No I don't work for Blizzard. When I did this, we were using a very fast custom wire format, but entirely hand-coded. Say flatbuffers without the code generation part. And having massive trouble with schema evolution.

Another group had already evaluated pb and found it way too slow. They had designed something similar but faster. I wrote my implementation of pb to prevent this, and showed pb could be fast enough. It was definitely the right choice at the time, as it gave us C++ speed close to the old format, plus easy interop with other languages.

> If protobuf works for Google then it essentially works for 99.999% of every other company on the globe.

Uh.. no. Google is a massive company, but if you browse the comments in this thread, you'll find multiple remarks like "this was built for Google's servers". Google have specific use cases, and they build software for that. The software may well be lacking for other use cases. I can totally imagine Blizzard wanting to write their own implementation, think of the benefits of reducing parse time in a multiplayer server.

I remember watching a GDC talk of JAM (https://www.gdcvault.com/play/1018184/Network-Serialization-...), which was Blizzard's solution to network serialization for WoW back in 2004. The talk even had performance comparisons against protobuf and it seemed like a decent alternative that worked well for them. Was this new alternative an extension of JAM, or a full rewrite? I know they have millions of players monthly, but I can't imagine saving a bit on bandwidth could have warranted a full rewrite, especially when the majority of their traffic is from their own datacenters.

Reply on news.ycombinator.com