You can't use signal without a cellphone... and cellphones are a privacy nightmare.
> your number is only visible to people who you've added to your address book as contacts
Means they already have my number.
There is still a tiny difference. I can pick a username and chat with people who have my phone number (without them knowing it is me). It's a very small difference.