2019 MBP, Big Sur: I have the same issue.

I have had better luck with the magic Trackpad plugged in via USB than over BT.