It's not their job to secure your card outside of their own system. Assuming these cards aren't stolen from within Amazon's own servers, it's up to you and your credit card to protect it from unauthorized use.
Yet there's this alternate universe where there are laws against facilitating a crime, receiving stolen property, etc. I'm not saying that sort of thing applies to Amazon, just saying isn't it weird that it doesn't?
They don't have a way of telling if the person calling is the actual owner of the card, or the scammer.
A traditional method used was to send two very small random payments, say 0.03 and 0.09 to the card in question, then ask the caller to confirm the payment details, how much etc, once they'd arrived. If they could provide these details that is pretty much as far as they can go in terms of verification.
And if the card has two authorized users, with the second person making purchases the first one doesn't know about? How is that at all Amazon's problem?