If you actually read the white paper on Apple's CSAM system, you'd know it is far more than just a boolean flag away. The system was specifically designed to prevent the kind of abuse everyone is worried about. I've been looking into the system deeply, and I fail to see how it could be co-opted for nefarious purposes without a complete redesign. Even in the most optimal attacks setup by a security researcher, I can't see how it gets anywhere near the worst-case scenario.
The worry seems to boil down to; "they could CHANGE it in the future". If the concern is about the potential for a future system, then such concerns are infinite, since infinite new systems could always be proposed or deployed. I would rather conserve my outrage for that moment, rather than be the boy who cried wolf.
Though this certainly isn't an argument for apathy. I think the amount of push-back Apple is getting is probably healthy, even if it's hyperbolic.