Sorry. There's no other option other than handing over your phone number which is a horrific idea. Just search SIM-swapping attacks (very common) or SS7 attacks (rare). You'll be surprised.

If you really want to use Signal, buy a throwaway SIM or probably use a Google Voice number to verify and login.

Honestly, I personally don't recommend using Signal unless they get rid of that phone number verification / login process. But I also don't blame general users for not using Element (Matrix) either (Which I think is more secure and works without a phone number, but it is unfriendly to use compared to Signal).