Google don't just have access to what you opt in to use. They also have access to whatever anyone else you communicate with effectively opts in to use on your behalf. For example, even if you don't want to give Google any data at all, there is a very high chance that they already have a lot of the emails you send via one of the other parties, and a fair chance that you won't even know it's happening unless you look at the MX records for their domain or something along those lines.

This is, IMHO, one of the biggest unresolved ethical issues with data protection and privacy today. On the one hand, it would be a great burden on society to stop everyone from using any third party services for anything involving personal data about someone else. On the other hand, when those services are no longer neutral and will process any data they receive for other purposes without the informed consent of some of the data subjects, that's a dubious practice at best. Another example is a social network's mobile app uploading your phone's entire contacts list, and just like that they've probably got a unique personal ID and a great deal of networking information about everyone on that list.