I don't get why these are automatically considered sinister. Measuring how people actually behave is an excellent way to improve your product. If this is done with sufficient care and anonymity, with no financial incentive to betray the user, I don't see the issue at all.
People don't like being watched. There's additional cognitive load in having to decide all the ways you might later regret giving that information away.
Once the information is out the door, it's out. You can't recall it. Apple or whatever company can have a change of leadership, mission or strategy that's incompatible with your values. But they still have your data and likely still can claim they have permission to use it.
For those of us who are older, it's also just discomfiting. We had at least a plausible illusion of privacy in our homes and our comings and goings. Now you have to take drastic and active steps to preserve privacy.
That's not to say I am fully against telemetry, but you asked why they're considered sinister. I think companies or organizations have a lot of work to do in order to comfort users.
It might be if done correctly but often telemetry is only used to justify existing wants. A typical example is removing a feature that isn't use a lot even though a) users might want the feature but don't even know about it because it is too hidden (which might even have been done intentionally as a first step preceding the telemetry-based removal) or b) might not be needed often but is absolutely essential in some cases. Like other statistics, telemetry can be used to justify almost any choice.
Meanwhile the same people using telemetry will happily ignore users trying to talk to them directly, including those users that make it clear that they don't want telemetry. So if you aren't going to listen to users why should anyone believe that your use of telemetry is going to be in those user's interest.
> If this is done with sufficient care and anonymity
And as a user you have no way of knowing that it is.
> with no financial incentive to betray the user
When is there not a financial incentive to betray the user? Any use where telemetry even makes sense is already a few to many relationship where users are at least somewhat replaceable. Data breaches are common but usually there are no real consequences for those that collected the leaked data even though they didn't really need it. Even intentional acts like selling the data will be forgotten soon enough, that is if they even come to light in the first place.
Even if a public company somehow managed to pass all of those caveats today the moment their leadership changed it would all go back out the window. And even a private company might go public or change leadership.