It's a little awkward, but Cmd-Opt-Shift-V is "Paste and Match Style" aka "paste as plain text".
Pro-tip (at least for me): control (^) doesn't change this, so you can just hold all the modifier keys rather than trying to remember which 3 let you paste as plain text.