> It's Facebook verifying that Facebook's other server is providing you the right key

Again, no :-). The third-party audit record is there such that... well... a third-party server can do that verification.

For WhatsApp it may be marginal because the client is proprietary (hence you can't audit it and verify that it actually uses the feature), but the Signal client is open source. So you could actually see that your Signal client checks the keys using third-party servers. That's something, I think.