Ask HN: How does Apple Maps know where I'm going?

This morning I got in my car to go to the gym, and on my phone's lock screen I could see a push notification from Apple Maps that read "15 mins to get to [gym_location]".

I'm very picky when it comes to which apps have access to my location, and Apple Maps' location access is set to "While Using" - but I _rarely_ use Apple Maps (maybe once every 6 months, by mistake). Further, if I ever did use Apple Maps at one time, I always quit any app after I'm finished using it.

So how does Apple Maps know where I'm going? Mind you, while I do tend to go to the gym on Saturdays, this morning was the earliest I've _ever_ been to the gym (by a long shot). If I've never used or had the app open in the background while going to the gym, how does Apple Maps know that I could even be _possibly_ going to the gym? Especially given how early it was in the morning?

I'm quite creeped out.