It is annoying, but it's not illogical.
If you have looked at something once, say there's a 99% chance that you have no further interest in it. But the 1% chance that you are potentially going to buy it after being shown the ad again is still much better odds than showing it to a random person, or a random person who is somehow correlated to the target market.
Even after seeing the ad 50 more times and ignoring it, you are probably still a statistically better lead than someone who has never interacted with the ad. They're not making a strong assumption about you, even though it seems that way. They're making a very weak assumption based on the small number of people who clicked once, hesitated, saw the ad 49 times more, carried on hesitating, and now are finally ready to buy.