This means there are specific, identifiable factors in your life that make it harder for you to exercise. Maybe you live far from a gym, or you don't have time during the week, or you have back pain that prevents you from exercising.
My point is that you're not going to do any progress until you make concrete changes based on your own specific situation.
So I guess my answer would be that it's hard because most people think it's a matter of motivation and self-control, whereas small, practical changes to your environment (for example, buying a bike) are actually much more effective.