Much more difficult to spoof an IP than a phone number. Once you’re behind a NAT, all bets are off, but the public IP has to be routable back to you in order to be usable.

Instead of phone number, think port on the phone company’s switch or a specific pair of wires. That’s hard to spoof.

It’s much easier to spoof a MAC address, which can identify specific hardware associated with the IP address, but that doesn’t make the IP any more difficult to identify.