The use of signatures in the US boggles my mind. Who checks this ever? I worked retail for years in the US and I would usually check the signature and (very rarely) challenge a signature. The customer was almost never happy I did this, so I generally wouldn't bother unless there were other reasons to be concerned. (For example, kids clearly using their parents card or someone who wrote "see ID" on the card.)
One additional thought: I now live in Europe, and use my contactless card for payments. However, sometimes it fails and I am required to use my chip and PIN. (I'm guessing because the bank thought the transaction needed additional security for some reason.) In this case, if I only had my phone, I would be stuck?