On several occasions I've seen ads appear based upon things being discussed, but never searched. Always with the realization I was near a smart device with a microphone (or an app with permissions)
I think in hindsight this will be similar to the realization that systems like XKeyscore and PRISM were not only technically possible but already deployed.
If i am not mistaken - this paper seems to look at explicit "Smart Speaker interactions" - not passive background listening which is what I believe you are alluding to.
Not arguing that I am saying that latter does/doesn't happen - but just that this paper is not proof of it. And there is a big difference.
So algorithms can percolate his interests to me after we meet and some time it would happen that we talked about it too.
Maybe not the best example (please chime in if you have a better one!), ha you ever shopped for a specific model of used car? Once you're on the hunt, suddenly you'll start seeing them everywhere when you're out and about, even though previously you didn't notice. There aren't more of that vehicle on the road than before, but now you're attuned to the pattern.
On top of that, it would be easily detected via networking monitoring.