For any larger project I would recommend working with a cargo workspace to compile as little as possible on each check/test/run/build.
Then you can build a DAG of crates and stick e.g. the Protobuf stuff in its own separate corner where it only needs to be recompiled on full rebuilds or when you work on it.
Feels a bit shitty to have to resort to managing crates instead of modules simply due to the compile times, but it is what it is.