I know it might seem odd, especially in the case of panhandlers, who may have difficulty persuading people that the transaction is secure in addition to the fact that some people will be judgmental and assume that if they can afford a smartphone, a card reader, and a bank account, they must not need the money.
But what's the real alternative? It sure isn't trying to persuade people to carry cash because of "The Man" or so that they can keep buskers and panhandlers in business. If people who earn money on the street can't accept card payments, they're screwed, plain and simple.
There might even be a non-profit startup idea here: something like Square, but optimized for panhandlers and buskers. Something that people get used to, that clearly communicates that the transaction will be secure and small (e.g. $1 or $2 every time), that charges absolutely minimal fees (or none if possible), and perhaps that works without a real bank account - e.g. money goes onto the device, and then the device itself can also be used to pay for things, like Apple Pay.