if you had verified fingerprints with Bob and are happily chatting with him, all the messages that reached him (two tick marks in WhatsApp) are safe.
Only those that have not yet been delivered (one tick mark) would, when the server sends you you a new key, be re-encrypted and re-sent.
All of this, as usual, is predicated on the client behaving as promised.
Boelter said: “[Some] might say that this vulnerability could only be abused to snoop on ‘single’ targeted messages, not entire conversations. This is not true if you consider that the WhatsApp server can just forward messages without sending the ‘message was received by recipient’ notification (or the double tick), which users might not notice. Using the retransmission vulnerability, the WhatsApp server can then later get a transcript of the whole conversation, not just a single message.”
I frankly didn't understand what was said here.
Then, at some point later, Eve on the compromised server could send a "oops, here's a new key, send everything undelivered again" message. Then, the client, as it is now, would just re-encrypt and re-send all those messages it deems undelivered so far (and then pop up the "key changed" message, if you had requested it in the settings).
You'd recognise the attack by seeing only single ticks on messages, even if Bob had seen them and answered.
If it can un-deliver all messages as well as have them re-sent with a new key then there may as well be no keys.