Yeah, dealing with user directory is a pain, and it is a pain because you can't make it completely functional (what about passwords). And it is the only place I found where nix doesn't work as you would expect it to in my 3 years of nixos use.
You can use agenix for secrets management. You can store all secrets encrypted with your SSH key in public config. The only non functional part is installing your SSH key, which is probably not a bad thing.