There’s a labyrinth of explicit user agreements, all of which are strictly enforced, and then there are further layers of voluntary restrictions on top of that.
Yes, Google might collect location data even when you’re not using Maps (there are a lot of passionate, dedicated Timeline users), but you have to have that enabled, and even then the data can only be used in certain ways.
Maybe they fail to use it competently but it definitely collects a lot of data and attempting to stop it results in punitive reactions and disabling of features not needing it - and repeated badgering to permit Google to collect data again.
Google also collects data that I was initially unaware that it is collecting.
Of course that opt-in requirement was only the result of a hard-fought class action settlement.
Take this with a grain of salt as my recollection of the litigation is somewhat hazy. But I believe Google argued, even when a user opted out of location tracking, Google could still keep the data for its own uses. Ie opting out of location tracking simply meant the user didn't have access to their location history, it didn't show up on the phone (couldn't be searched by suspicious spouses) etc.