Yeah, it's a big flaw in the setup. The instructions I linked have users add it temporarily, to unlock the rest of the options, and remove it afterwards.
Once you've added it, it's google's forever and you've only got their word that they've 'deleted' it. I bet it's still available with a warrant.
The problem isn't that Google knows your phone number but that they will use it (or more precisely the attacker will use it after redirecting your texts) for account recovery which you can prevent by removing the number.
Yes, and Facebook and Dropbox have enabled Yubikey/U2F support in the same way. It's stupid and it gives you a false sense of security (because you're using hardware tokens, which you'd expect to be more secure).
On Facebook, you can subsequently take your phone number off.
Only an Id10t would think it's actually deleted from FB-data logs.
The point is that it's no longer an attack vector. Wrt privacy, sure, it's probably still there.