You can forward someone else's mail too. Just go to the post office, fill out the form, and drop it in the mailbox. There's no verification, though there is notification at both the old and the new address. Not to mention it's a crime to do that.
Personally I have this turned on, because I'm conscious that every piece of information you'd need to send a fake address change notification for me is either public or easily obtainable with a call to the Tax Agency, which is far from reassuring.
It's not great for privacy though, so I understand the resistance of the US and the UK to it. Mind you, the US wouldn't have to make the whole thing public like Sweden does for… reasons.
Cue the fever dreams whenever anyone merely suggests we harmonize, normalize the details. As if predictable standards for signature verification will inevitably lead to a military coup.
*Asylum seekers and various types of temporary residents and such are not technically considered to be in the population register, though they are usually still in fact in the database, just with less information and a flag saying they don't count
I haven't personally had a social security card in many decades. For the first time in I don't know how long, I do need to provide some form of proof of SSN to get a RealID drivers license renewal but last year's W2 is sufficient for that.
That's my experience after hand delivering the form to the SS office so it would be quick.