Kaitai, binary format parsing web IDE

There's the small stuff: YAML is convenient as a universal language, but it means that declarations are very verbose - you can't use any sort of syntax sugar, you're having l basically writing the underlying AST that a more convenient language would parse to. It's also still YAML, with all the problems that come from that.

I also can't make head nor tail of the IDE. The big selling point is that it can parse test data you provide automatically as you build your Kaitai structure, but you can't edit the test data at all, and last I checked it didn't provide any sort of completion or hinting as you write your parser. So I might as well edit the structure in my editor of choice and occasionally copy it into the IDE if there's something I need to check, except then I also need to generate an interesting binary test case file and upload it, because, like I said, I can't edit those test cases in the IDE.

But the most important thing is that it literally only does parsing. Kaitai is a poor way to describe a binary API because if you do that, you'll get parsers for free, but you'll have to write all the generating code yourself - at which point, you have to wonder what advantage Kaitai is actually giving you. As I understand it, this isn't some missing feature that's in the works - the Kaitai team do not want to make this a two-way format, that is a complete non-goal.

Which means, as far as I can understand, the main purpose of Kaitai is to parse binary file formats like .PNG or .PDF files without the need for a special parser. It's basically a binary PEG (but, as mentioned before, in YAML).

Parsing this data allows us to build Grafana powered dashboards (https://dashboard.satnogs.org, click on the list of active satellites).

Certain parts of the binary I need to parse are compressed using some custom algorithm. I do have an implementation for compression and de-compression in C++. Is it possible to provide this to Kaitai? Maybe via some C FFI mechanism. From the looks of it, Python Construct can cover this; however, I'd really like to use Kaitai's IDE and therefore Kaitai directly.

Kaitai is like protobuf, but for binary formats. It's an incredibly clever way to not write n*m-many parsers for each language. You write a spec for your binary format and the Kaitai libraries in each language can then decode (and presumably encode) the payloads. Gif, wav, pdf, zip, fbx, your custom format, whatever.

This web IDE makes it easy to visualize and develop parser specs against actual data.

Absolutely brilliant ideas!