The fraudulent message asks for a yes or no reply but does not care about the answer spefically; only that there was an answer. So the victims are the people who couldn't ignore the message and had to say no. Most likely the people who say yes are still taken into account , because they confirmed there was a person behind the number. They'll get a new scam later on.
But the people saying no are the target.
A lot of people feel bad if they don't answer the phone, or a message, or the doorbell. So you prey on their niceness.
How to fight back? Don't. The way to defeat the scam is to not acknowledge it.
Ignore whatever primal urge you have to get involved, or teach them whippersnappers a lesson, and cast it into the void.