Then I tested it with test data streams generated by the reference C++ implementation and Python implementation.
It's actually kind of a pain, because the C++ serializer generates a variety of different data types depending on the values you are packing, not the type of the values you are packing. Let's say I encode a uint32_t field. The stream might get a uint8, a uint16, a uint32, or even an int32 (for reasons that completely elude me).
Also, C++ strings come out as the 'ext' type while Python strings come out as the 'string' type, so I have to accommodate both, even though they are both basically byte strings.
So, I want to tell the deserializer what kind of output field I'm expecting, for each struct member or array, and then look in the data stream to see if there is a data object there that came from the same type. But this is impossible, so the per-type decode functions have to be quite complicated to handle a variety of types.
So - it works but I can't do much on the deserialization side to verify that the data I'm unpacking really matches what was encoded. I can only detect very broken cases, like when I'm unpacking a uint32_t and in the stream there is an int32_t with a negative value.
I guess this is mostly done for optimization, but two things would make it a lot better:
- if the spec actually specified how data types in different languages were allowed to be encoded
- if the encoded data contained _two_ type fields, one indicating the original source data type and another indicating the type it was encoded into.
Basically the spec is just way too "loose" to make it usable for the use case I'm trying to use it for, which is to easily generate data that is sent to a micro and stored in EEPROM, then deserialized out of EEPROM later.
That's probably not very close to a use case the original designer had in mind. But I haven't found anything that works better (less decode logic).