for the overwhelming majority the issue is raising a kid, not conceiving one. And given Japan's fairly unique aversion for kids outside marriage (I think it's like 2% compared to 40% in the US or Europe), this is almost completely cultural.
If you're facing the choice of not having kids or being delegated to married housewife a lot of women across East-Asia just choose to not want one. Understandable, really.