That would be waiting for a quantum computer and quietly hoping that a) nobody develops a strong enough post-quantum scheme and b) there is still civilization after RSA and ECC are broken? Correct me if I'm wrong.
NIST standardized Kyber and Dilithium, and for now at least, they seem to be holding up. I'd still want to do hybrid (ECC+PQ) asymmetric crypto for the time being, but we're (slowly) starting to gain a modicum of confidence in the new standards, enough for deployment
There's also the question of, if you can distribute a key which is at least the same size as your message over a secure channel - why not just distribute your message over that channel in the first place?
Latency? You can hand deliver a password ahead of time, but not messages.
You still have to reliably move a chunk of out-of-band information in a way such that it gets to (and only gets to) the person you want to have it.
Also, it isn't just a "chunk", for one-time pad it has to be the same length as the messages. Which is fine if just short messages but a lot harder if lots of data.
If can exchange lots of data, better off using them as keys for stream cipher.
But the question of, "Why not just send the message instead of the pad" is pretty straightforward: when you have the opportunity to safely deliver the pad, you don't know what the message will be. When you do know what the message will be, you don't have the opportunity to safely deliver the pad.
I read some years ago about a non quantum technique to achieve the same based on (I think) noise in a coupled electronic system. I wonder if that has been tested further.
Eg, rolling a dice is deterministic, and I imagine an algorithm exists that could recover the value of a dice throw from a recording of the sound of it rolling and it's initial position. But once that sound has turned into heat, and that heat has conducted itself about the walls and into the air, I don't think it's possible to recover the sound.
See also:
"Is flipping a coin random?" (Numberphile)
There’s nothing in the laws of physics that prohibits us turning burned paper smoke back into a document and recover the information.
When you burn a document, all the matter might be transferred into the smoke, but you've rendered it into a stream of particles which is small enough to be effected by Brownian motion. Reversing the process (figuring out the initial position of each soot particle) involves knowing the position and momentum of the air molecules impacting the soot particles. In principle, you could take the current position and momentum of those particles and extrapolate backwards - but you can't actually measure that, not even in theory.
It's like people read that OTPs are the only encryption method that has been proven to be completely unbreakable (when used correctly) and stop reading there, and then completely miss all the things OTPs don't solve (ie, guaranteeing authenticity), not to mention their massive glaring limitation: How do you transfer the encryption key?