* Men aren't directly affected by it (~50% population)
* Woman over 40 aren't generally affected by it
So woman between 18-40 who can vote are the group most affected by abortion policy. And as you point out, even they aren't directly affected until they actually need one. So the skin-in-the-game for most people is very low. Most people vote and are opinionated on it as a sort of proxy for woman's rights.
However, some issues like house affordability, crime, employment, etc are very high for skin-in-the-game. People are currently affected or know people currently affected by these issues.