I stopped caring about this by having home-manager do all this for me. I don't really care where the files are anymore because I change them through Nix, not directly, and I can choose where that is located.