Hacker News Reader

Comment by hansonkd13 | Hacker News Reader

Author here, combining Redis protocol and MsgPack is a much more platform independent way of building a protocol.

The redis protocol and Msppack are both only a a hundred lines or so for a parser. Meaning you can build your own from scratch in a new language if one isn’t supported.

Its also stupid fast.

Compared to protocol buffers which can be extremely complicated to grok on the binary level.

I built a more robust API RPC for Python here based on Redis and MsgPack: https://github.com/hansonkd/tino

View on HN

NATS has a similar protocol to Redis and optimized for this use cause of doing pub/sub based low latency Request/Response instead of HTTP. The payload is opaque so you can use msgpack if needed, and the protocol also supports sending headers now: https://docs.nats.io/nats-protocol/nats-protocol#protocol-me...

(disclaimer: I'm one of the NATS maintainers :) )

> extremely complicated to grok on the binary level.

That seems like a weird claim. Protobuf on the wire only has four types: short fixed-length values, long fixed-length values, variable-length values with continuation-bit encoding, and length-prefixed values, where the length is continuation-bit encoded.

Msgpack has 37 different wire types!

No use in changing things if it's working for you, but you might be interested in trying out msgspec (https://jcristharif.com/msgspec/) in tino instead of using msgpack-python. The msgpack encoder/decoder is faster, and it comes with structured type validation (like pydantic) for no extra overhead (https://jcristharif.com/msgspec/benchmarks.html).

Did you consider ASN.1 for the protocol?

Lol. No offense, but I did a report in uni comparing a few alternatives (asn.1, json/xml over http, protobuf v1 (it's a while ago).

And on one hand, sure asn.1 exists, on the other there's been some issues in for example nfs. I don't think you want asn.1 today - but maybe cap'n'proto.

But i think redis is probably an interesting approach.

Not sure how many of these beyond whatever/http2 have a sane pipelining+authenticated encryption story? I suppose you could "dictate" security at the ip level via vpn/wireguard or something. Or use Unix sockets.

What are the problems with ASN.1? It seems to work in 5G and LTE just fine? ASN.1 is simply a way to have portable data structures — are your problems with ASN.1 that it cannot effectively implement portable data structures or is your problem with the systems that have been built upon ASN.1?

I seem to recall the issue was with the tooling around generating clients and servers and preserving type information - but it's been almost 20 years and I don't have my notes on hand.

I also think there were some nfs security issues relating to asn.1

Might have been something along the lines of this:

https://www.infosecmatter.com/metasploit-module-library/?mm=...

I guess that (and related errors in openssl and bouncy castle) might not be due to problems with asn.1 per se, but rather difficulty in writing safe libraries for parsing in C.

Reply on news.ycombinator.com