Thanks, but Xfce's startup applications list (which is just a frontend for ~/.config/autostart) is already executed after the X server starts, and after .xinitrc.
(Apologies if me saying 'runs at startup' was misleading -- obviously if I had tried to do it from a script that ran on boot, before X starts (rather than on session start) it could never have worked in the first place!).
Edit: now I've started thinking about it again, I've changed the autostart script to call another which sleeps for 10s before xmodmap, which seems to have fixed the problem.