Comment by NavinF | Hacker News Reader

Comment by NavinF | Hacker News Reader

Parent Full thread

> didn’t find any standard for separating protobuf messages

The fact that protobufs are not self-delimiting is an endless source of frustration, but I know of 2 standards for doing this:

- SerializeDelimited* is part of the protobuf library: https://github.com/protocolbuffers/protobuf/blob/main/src/go...

- Riegeli is "a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding": https://github.com/google/riegeli

https://msgpack.org/

Huh? This is an entirely different serialization format altogether.

Just stick this in front of your frames:

    81 A2 70 62 C6 <32 bit be length> <protobuf data>

You got me. What's that format / magic number? At first I thought it was the zlib header for incompressible data, but that's not right. Is that just the MAC address of some NIC you own?

It's msgpack:

0x8n A map containing "n" key/value pairs -> 0x81 introduces a single key/value pair

b101XXXXX : A string where XXXXX is the string length -> 0xA2 introudces a 2 byte string

0xC6 followed by a 32-bit value N introduces an array of bytes of size N

0x70 0x62 "pb"

So it roughly corresponds to

  {"pb": <protobuf data> }

Poster is being cute.

Create a MessagePack message containing a byte array, put the protobuf in that. Of course, the enterprise solution would be to hex encode it and put it in a CDATA section in an XML document.

Doesn't have a Brainf*k implementation, disqualified.

/s

Reply on news.ycombinator.com