Them knowing what your favourite coffee shop is or where you work and live is just the tip of the iceberg.
I used to be a very heavy and trusting Google user. It started to get a little bit uncomfortable when I noticed an alert on my phone notifying me that there was a restaurant nearby that I might like to try. It occurred when I had 30-40 minutes to spare, it was a restaurant that indeed looked interesting to me, and was within a 5 minute walk. Yes that's useful but my conclusion was that this company knows way too much about me. It didn't make me comfortable when Google alerted me about flights I never explicitly told it that I was taking, and offered suggestions about how to get to the Airport that suited my routine.