Using amoral implies there is at least a consensus on what is moral and they are acting opposite to it. This is an entire conversation on what is even moral to begin with. It is smearing someone by associating your "morality" to a particularly grey area.