Do you have a repo/gist where you can throw up a sanitized rpi config? I don't know much about Nix but what you are describing sounds like a nice use case as I do have to periodically rebuild different RPIs.
The setup is mostly mirrored from https://sgt.hootr.club/molten-matter/nix-distributed-builds/
Be sure to enable binfmt for whatever architecture your pi uses on your main machine for cross compilation.
For generating the iso I used nixos-generators.
https://github.com/NelsonJeppesen/nix-lifestyle/blob/main/ni...
After the first hump of learning the Nix ecosystem, the rewards on time are great
EDIT: After looking over the repo, I can see how much code would be unneeded over a Nix[os] configuration