I cannot say I have ever encountered any of the problems you describe and I use Apple Pay daily to pay for everything. Just double tap the power button before paying and hold it to the terminal, it takes like 2 seconds.
I double-tap and Face ID scan right in front of me first, then I hold it out and tap on the reader. It always works and there's hardly any delay. Fishing my credit card out of my back wallet takes 2 seconds longer.
That said, nowadays I use my Apple Watch and that's even faster -- double click the side button, and tap. (No Face ID at all)
1. Long hair can sometimes get in my face, making FaceID unreliable
2. Camera angle in the "tap" position is not the angle it needs to be in for the FaceID unlock. I often double click then try to tap my phone, only to remember that I need to FaceID unlock.
Also two things: if you’re willing to accept a less secure Face ID you can turn off attention (1). And if you’re paying for transit, you can turn on Express transit if supported on your provider — then it’s just tap, no need to auth.
(1) https://support.apple.com/guide/iphone/face-id-attention-iph...
Great when you’re in self-checkout in a busy supermarket…
1. grab wallet from pocket, find card, align card to tap to pay target
2. grab phone from pocket, double tap phone side button while looking at it, align phone to tap to pay target
It also depends what you already have out. If you already have your wallet or phone out (wherever your store loyalty card is), that mode (wallet or phone) is that much easier