Reverse Engineering Protobuf Definitions from Compiled Binaries
arkadiyt.com
arkadiyt.com
It may be bit-rotted by now, but a couple of years ago I hacked together a python script that would extract a proto definition from Objective C apps by scanning the assembly output and looking for the patterns of code generated by the protoc compiler. I put it on github in case it is useful to somebody.
https://gist.github.com/dunhamsteve/224e26a7f56689c33cea4f0f...
Compiled ObjectiveC is a bit simpler than compiled C/C++, you can read the method invocations out of it. I haven't looked into how hard the output of Swift is to read.
I've also analyzed protobuf data (Apple Notes) by writing code that decodes the data in a generic fashion and output a guess at the schema. I would run on about 100 samples, to help distinguish binary data from sub-objects, to detect optional fields, and to detect 'repeated' fields. Then you have to go through and figure out what all of the fields are.
I succeeded, but later learned that the notes web app embedded the plain text .proto file, which would have made things a lot easier.
I’ve written ProtobufDecoder which takes a different approach: analyze the structure of the actual messages to help you figure out the protobuf structure of a message.
* https://github.com/grpc/grpc-go/blob/master/Documentation/se...
* https://chromium.googlesource.com/external/github.com/grpc/g...
You must explicitly enable it in code and many people… well just dont
See also https://github.com/grpc/grpc/blob/master/doc/server-reflecti...
The primary difference between what grpc does and what stubby does is that grpc uses a stream to ensure that the reflection requests all go to the same server to avoid incompatible version skew and duplicate proto transmissions. With that said, in practice version skew is rarely a problem for grpc_cli style "issue a single RPC" usecases: even if requests do go to two or more different versions of a binary that might have incompatible proto graphs, it is very common for the request and response and RPC to all be in the same proto file so you only need to make one RPC in the first place unless you're using an extension mechanism like proto2 extensions or google.protobuf.Any.
https://github.com/protocolbuffers/protobuf/blob/main/src/go...
Xgooglers trying to make do on the cheap will just create a Union of all their messages and include the message def in a self-describing message pattern. Super-sensitive network I/O can elide the message def (empty buffer) and any for RecordIO clone well file compression takes care of the definition.
Definitely useful to be able to dig out old defs but protobuf maintainers have surprisingly added useful features so you don’t have to.
Bonus points tho for extracting the protobuf defs that e.g. Apple bakes into their binaries.