The issue isn't that the other party knows your IP, it's that this is an unexpected connection for no other purpose than to obtain your public IP, in addition to your LAN IP.
There is not even a connection. During the RTCPeerConnection's "ICE Gathering" process local IPs are discovered and, if STUN/TURN servers are configured within the RTCPeerConnection, STUN requests are sent to those servers (which help retrieving the public IP of the computer/router). But you don't need even to send a single packet in order to get local IPs (private ones, VPN ones, etc).