On the one hand, I sorta agree with you and I find some of the kneejerk anti-telemetry perspectives to be maybe overly paranoid. On the other hand, I do think there's a valid concern here because the telemetry is closed source.

Short of reverse engineering the official binaries and/or inspecting what it sends back to Microsoft, there's no way of knowing what it's actually collecting, which is understandably concerning given Microsoft's history. And you'd have to do that for every release, since any update could expand the scope of what they gather.

Personally, I don't quite care enough to bother doing something about it, but I can see why folks who are inclined to distrust Microsoft would not want to trust them on this.