That's true. If this is an issue you could avoid using a signaling server altogether, and instead exchange offer/answer via e.g. GPG-encrypted email.

I wonder if there are any designs for a signaling server/system that doesn't have this loss of privacy of connection information, or if you just very quickly end up reimplementing Tor when you try..