I read somewhere that it was an early-batch problem and that it's been fixed in newer iterations of the Pro, but there's no way I'm going to spend money on them again just to find out. The AirPods Pro are the only Apple device I've ever been totally and completely disappointed with. The regular AirPods remain one of the most unambiguously-good Apple devices I've bought.
The workaround I discovered was that if I plugged in my bluetooth trackpad via USB during a call, these issues would basically never happen.
Not to mention all the other general issues with them failing to stay connected to my Mac, or failing to connect initially. These issues also happened on multiple MacBook Pros, including new top-of-the-line 16" models. Whereas I have basically zero problems with AirPods on my iPhone or AppleTV.
In short: the macOS bluetooth stack has some serious shortcomings that aren't present in iOS.
The most reliable Bluetooth I've found is a hardware device that pretends to be a USB sound card and connects to my headphones via Bluetooth. Anything with a full operating system seems like a disaster.