How will they be able to link your phone number to your username? They can't do anything with your number unless you choose to.
By "link" I mean they immediately know what person the username belongs to iff they already had that person's phone number because the chat that is initialized after they scan the QR code is just the old chat being continued.
Exactly. I think that's important to know before people start giving out their Signal handles left and right because they think they're anonymous now.