But it could also just be that I never started to like exercising in the first place. I have myopia since early childhood - so from the first years in school, glasses gave me logistical problems on PE lessons. It was especially problematic during team games - when I took the glass off, I was the worst player in class. When I left them on, I could play somewhat well, but I was risking breaking them when hit by a ball (it actually happened). So I ended up avoiding team sports as much as I could, and there a negative feedback loop starts (not playing made me only suck more and be less willing to play).
Anyway, thanks for the advice :).