The difference is that a reader isn’t in any way a victim. They’re choosing to read a book. And if they don’t pay, the writer will often simply be paid less, to the tune of the royalties on that one book. So, yes, that is stiffing someone.

If everyone opted out you could force major change, sure, but in that case you shouldn’t be reading the book. That’s a true boycott. Reading without paying isn’t principled - it’s just cheap. And if you don’t actually organise it achieves nothing - except stiffing the author.