I've noticed the same thing. It's the delay between waking up and freezing that makes it so frustrating.

I'm using Pop_OS on an AMD 7950x + Nvidia RTX 4080. I'm using the proprietary Nvidia drivers, so I blamed that initially, but I see another comment mentioning that this only happens on their AMD systems, so maybe they are not the culprit this time.

The only solution I've found is to disable sleep/suspend entirely.