Hacker News Reader

Comment by mgaunard | Hacker News Reader

It's 2026 and I'm still defining my own messaging and wire protocols.

Plain C structs that fit in a UDP datagram that you can reinterpret_cast from is still best. You can still provide schemas and UUIDs for that, and dynamically transcode to JSON or whatever.

View on HN

Until you have to work with big and little endian systems. There are other weirdness about how different computers represent things as well. utf-8 / ucs-16 strings (or other code pages). Not all floats are ieee-754. Still when you can ignore all those issues what you did is really easy and often works.

I disagree. Big endian is long dead and not worth worrying about. And code pages too. What is more important, is dealing with schema changes, when you add new fields to requests and responses.

There are niches where those matter.

but yes schema changes is most likely to get you today

Provided that:

    - you agree never to care about endianness (can probably get away with this now)

    - you don't want to represent anything complicated or variable length, including strings

You can have strings by using relative pointers ("string starts 123 bytes before this").

You can also just use an array which sets a max capacity, and either use a null-terminator or a separate size field.

In practice you probably want to have both, and choose what's most practical based on the message.

If you decide to use UDP, do you ignore the transmission errors or write the handling layer on your own?

I handle it in different ways by topic.

For topics which are sending the state of something, a gap naturally self-recovers so long as you keep sending the state even if it doesn't change.

For message buses that need to be incremental, you need to have a separate snapshot system to recover state. That's usually pretty rare outside of things like order books (I work in low-latency trading).

For requests/response, I find it's better to tell the requester their request was not received rather than transparently re-send it, since by the time you re-send it it might be stale already. So what I do at the protocol level is just have ack logic, but no retransmit. Also it's datagram-oriented rather than byte-oriented, so overall much nicer guarantees than TCP (so long as all your messages fit in one UDP payload).

What you use is perfect for short-range communication (application and child process talking over shared memory), but not good for long-range communication (over Internet) because you can have old client talking to new version of a server, so you will have to add version numbers and have the code to parse outdated formats. But protobuf has compatibility built in and you do not need to write anything to support outdated clients. Also, protobuf uses solutions like varints to compress data to use less network traffic. So it is obviously made for long-range communication, and you probably do not have that and send 7 zeros for every small number.

TL;DR protobuf has version compatibility and compact number encoding.

I already said you can UUIDs and schemas, and even dynamic conversion between mismatched schemas.

Doing plain C structs doesn't prevent any of this.

It requires extra effort to write conversion algorithm for older data structure version.

The converter is generated automatically based on the differences between the two schemas.

Takes zero effort other than CPU cycles.

Reply on news.ycombinator.com